* [Xenomai-core] Xenomai on PXA @ 2006-07-07 15:02 Danilo Levantesi 2006-07-07 20:32 ` Gilles Chanteperdrix 0 siblings, 1 reply; 36+ messages in thread From: Danilo Levantesi @ 2006-07-07 15:02 UTC (permalink / raw) To: xenomai [-- Attachment #1: Type: text/plain, Size: 610 bytes --] Hi to all. I should study a soft real time linux kernel for a university project on an arm architecture (in detail I have got an Ipaq 3970 with a xscale processor) I have a working distro based on a 2.4.19 vanilla kernel for that PDA. Then i tried to apply the RTAI patch without success and on rtai-ml they told me arm is no longer supported and to have a look at xenomai. I read on the xenomai-ml archive and found only 2 posts (from Bart Jonkers) about pxa port, without answers. So what is the status of the port? Is anything working on a pxa arch? Many thanks in advance. Danilo Levantesi [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-07 15:02 [Xenomai-core] Xenomai on PXA Danilo Levantesi @ 2006-07-07 20:32 ` Gilles Chanteperdrix 2006-07-11 6:20 ` Detlef Vollmann 0 siblings, 1 reply; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-07-07 20:32 UTC (permalink / raw) To: vagninu; +Cc: xenomai Danilo Levantesi wrote: > Hi to all. > I should study a soft real time linux kernel for a university project on an > arm architecture (in detail I have got an Ipaq 3970 with a xscale processor) > I have a working distro based on a 2.4.19 vanilla kernel for that PDA. > Then i tried to apply the RTAI patch without success and on rtai-ml they told > me arm is no longer supported and to have a look at xenomai. > I read on the xenomai-ml archive and found only 2 posts (from Bart Jonkers) > about pxa port, without answers. > So what is the status of the port? Is anything working on a pxa arch? Bart questions got answered, in the thread starting at : https://mail.gna.org/public/xenomai-core/2006-05/msg00149.html More recently, Detlef Vollmann seem also to have begun a port to PXA. The mailing list thread started in june at: https://mail.gna.org/public/xenomai-core/2006-06/msg00222.html and continued in july at: https://mail.gna.org/public/xenomai-core/2006-07/msg00003.html I do not know about the status of their work, but maybe they will be willing to give a more detailed answer. -- Gilles Chanteperdrix. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-07 20:32 ` Gilles Chanteperdrix @ 2006-07-11 6:20 ` Detlef Vollmann 2006-07-11 15:01 ` Stelian Pop ` (2 more replies) 0 siblings, 3 replies; 36+ messages in thread From: Detlef Vollmann @ 2006-07-11 6:20 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai [-- Attachment #1: Type: text/plain, Size: 871 bytes --] Gilles Chanteperdrix wrote: > Danilo Levantesi wrote: > > So what is the status of the port? Is anything working on a pxa arch? > > More recently, Detlef Vollmann seem also to have begun a port to > PXA. Unfortunately my priorities were shifted, so I'm not able to work on it for the next few weeks. What I've done so far is attached. IT'S UNTESTED!!! So it's nothing you can just use, but something that you can build on (possibly). I'd be happy to hear about any experience with this. What is missing is a look at entry-macro.S. Stelian Pop has done something for the Integrator that I don't really understand and therefore I can't say whether the PXA needs something similar. Detlef -- Detlef Vollmann vollmann engineering gmbh Linux and C++ for Embedded Systems http://www.vollmann.ch/ Linux for PXA270 Colibri module: http://www.vollmann.ch/en/colibri/ [-- Attachment #2: time.c --] [-- Type: text/plain, Size: 8226 bytes --] /* * arch/arm/mach-pxa/time.c * * Author: Nicolas Pitre * Created: Jun 15, 2001 * Copyright: MontaVista Software Inc. * * Preliminary support for I-PIPE !!! COMPLETELY UNTESTED !!! * Copyright (c) 2006 Detlef Vollmann <dv@domain.hid> * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2 as * published by the Free Software Foundation. */ #include <linux/config.h> #include <linux/kernel.h> #include <linux/module.h> #include <linux/init.h> #include <linux/delay.h> #include <linux/interrupt.h> #include <linux/time.h> #include <linux/signal.h> #include <linux/errno.h> #include <linux/sched.h> #include <asm/system.h> #include <asm/hardware.h> #include <asm/io.h> #include <asm/leds.h> #include <asm/irq.h> #include <asm/mach/irq.h> #include <asm/mach/time.h> #include <asm/arch/pxa-regs.h> #ifdef CONFIG_IPIPE #ifdef CONFIG_NO_IDLE_HZ #error "dynamic tick timer not yet supported with IPIPE" #endif int __ipipe_mach_timerint = IRQ_OST0; static DEFINE_SPINLOCK(timer_lock); int __ipipe_mach_timerstolen = 0; EXPORT_SYMBOL(__ipipe_mach_timerstolen); unsigned int __ipipe_mach_ticks_per_jiffy = CLOCK_TICK_RATE / HZ; EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); #endif static unsigned long timer_reload = LATCH; static int timer_initialized = 0; static inline unsigned long pxa_get_rtc_time(void) { return RCNR; } static int pxa_set_rtc(void) { unsigned long current_time = xtime.tv_sec; if (RTSR & RTSR_ALE) { /* make sure not to forward the clock over an alarm */ unsigned long alarm = RTAR; if (current_time >= alarm && alarm >= RCNR) return -ERESTARTSYS; } RCNR = current_time; return 0; } /* IRQs are disabled before entering here from do_gettimeofday() */ static unsigned long pxa_gettimeoffset (void) { long ticks_to_match, elapsed, usec; if (!timer_initialized) return 0; /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; /* We need elapsed ticks since last match */ elapsed = LATCH - ticks_to_match; /* don't get fooled by the workaround in pxa_timer_interrupt() */ if (elapsed <= 0) return 0; /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; return usec; } #ifdef CONFIG_NO_IDLE_HZ static unsigned long initial_match; static int match_posponed; #endif static irqreturn_t pxa_timer_interrupt(int irq, void *dev_id, struct pt_regs *regs) { int next_match; write_seqlock(&xtime_lock); #ifdef CONFIG_NO_IDLE_HZ if (match_posponed) { match_posponed = 0; OSMR0 = initial_match; } #endif /* Loop until we get ahead of the free running timer. * This ensures an exact clock tick count and time accuracy. * Since IRQs are disabled at this point, coherence between * lost_ticks(updated in do_timer()) and the match reg value is * ensured, hence we can use do_gettimeofday() from interrupt * handlers. * * HACK ALERT: it seems that the PXA timer regs aren't updated right * away in all cases when a write occurs. We therefore compare with * 8 instead of 0 in the while() condition below to avoid missing a * match if OSCR has already reached the next OSMR value. * Experience has shown that up to 6 ticks are needed to work around * this problem, but let's use 8 to be conservative. Note that this * affect things only when the timer IRQ has been delayed by nearly * exactly one tick period which should be a pretty rare event. */ #ifdef CONFIG_IPIPE /* * - if Linux is running natively (no ipipe), ack and reprogram the timer * - if Linux is running under ipipe, but it still has the control over * the timer (no Xenomai for example), then reprogram the timer (ipipe * has already acked it) * - if some other domain has taken over the timer, then do nothing * (ipipe has acked it, and the other domain has reprogramed it) */ if (!__ipipe_mach_timerstolen) { #endif do { timer_tick(regs); #ifndef CONFIG_IPIPE OSSR = OSSR_M0; /* Clear match on timer 0 */ #endif next_match = (OSMR0 += timer_reload); } while( (signed long)(next_match - OSCR) <= 8 ); #ifdef CONFIG_IPIPE } else { timer_tick(regs); } #endif write_sequnlock(&xtime_lock); return IRQ_HANDLED; } static struct irqaction pxa_timer_irq = { .name = "PXA Timer Tick", .flags = SA_INTERRUPT | SA_TIMER, .handler = pxa_timer_interrupt, }; static void __init pxa_timer_init(void) { struct timespec tv; set_rtc = pxa_set_rtc; tv.tv_nsec = 0; tv.tv_sec = pxa_get_rtc_time(); do_settimeofday(&tv); OIER = 0; /* disable any timer interrupts */ OSCR = LATCH*2; /* push OSCR out of the way */ OSMR0 = LATCH; /* set initial match */ OSSR = 0xf; /* clear status on all timers */ setup_irq(IRQ_OST0, &pxa_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ timer_initialized = 1; } #ifdef CONFIG_NO_IDLE_HZ static int pxa_dyn_tick_enable_disable(void) { /* nothing to do */ return 0; } static void pxa_dyn_tick_reprogram(unsigned long ticks) { if (ticks > 1) { initial_match = OSMR0; OSMR0 = initial_match + ticks * LATCH; match_posponed = 1; } } static irqreturn_t pxa_dyn_tick_handler(int irq, void *dev_id, struct pt_regs *regs) { if (match_posponed) { match_posponed = 0; OSMR0 = initial_match; if ( (signed long)(initial_match - OSCR) <= 8 ) return pxa_timer_interrupt(irq, dev_id, regs); } return IRQ_NONE; } static struct dyn_tick_timer pxa_dyn_tick = { .enable = pxa_dyn_tick_enable_disable, .disable = pxa_dyn_tick_enable_disable, .reprogram = pxa_dyn_tick_reprogram, .handler = pxa_dyn_tick_handler, }; #endif #ifdef CONFIG_PM static unsigned long osmr[4], oier; static void pxa_timer_suspend(void) { osmr[0] = OSMR0; osmr[1] = OSMR1; osmr[2] = OSMR2; osmr[3] = OSMR3; oier = OIER; } static void pxa_timer_resume(void) { OSMR0 = osmr[0]; OSMR1 = osmr[1]; OSMR2 = osmr[2]; OSMR3 = osmr[3]; OIER = oier; /* * OSMR0 is the system timer: make sure OSCR is sufficiently behind */ OSCR = OSMR0 - LATCH; } #else #define pxa_timer_suspend NULL #define pxa_timer_resume NULL #endif struct sys_timer pxa_timer = { .init = pxa_timer_init, .suspend = pxa_timer_suspend, .resume = pxa_timer_resume, .offset = pxa_gettimeoffset, #ifdef CONFIG_NO_IDLE_HZ .dyn_tick = &pxa_dyn_tick, #endif }; #ifdef CONFIG_IPIPE void __ipipe_mach_acktimer(void) { OSSR = OSSR_M0; /* Clear match on timer 0 */ } unsigned long long __ipipe_mach_get_tsc(void) { if (!timer_initialized) return 0; else return OSCR; } EXPORT_SYMBOL(__ipipe_mach_get_tsc); /* * Reprogram the timer */ #if 0 /* old nonsense code, left here for reference */ static inline void set_dec(unsigned long delay) { int next_match = OSCR + delay; timer_reload = delay; if (delay < 8) { OSSR = OSSR_M0; /* Ack the timer in case it fired in the meantime */ ipipe_trigger_irq(__ipipe_mach_timerint); return; } else OSMR0 = next_match; if (OSCR < OSMR0) return; /* everything is fine, we'll get an interrupt */ if ((signed long)(next_match - OSCR) <= 6) { /* two ticks are gone, so we got interrupted */ /* we do a busy wait here, just to play save */ while (OSCR < next_match) { if (OSCR < OSMR0) return; } OSSR = OSSR_M0; /* Ack the timer in case it fired in the meantime */ ipipe_trigger_irq(__ipipe_mach_timerint); } } #endif void __ipipe_mach_set_dec(unsigned long delay) { unsigned long flags; int next_match = OSCR + delay; spin_lock_irqsave_hw(&timer_lock, flags); timer_reload = delay; if (delay < 8) { OSSR = OSSR_M0; /* Ack the timer in case it fired in the meantime */ ipipe_trigger_irq(__ipipe_mach_timerint); return; } else OSMR0 = next_match; spin_unlock_irqrestore_hw(&timer_lock, flags); } EXPORT_SYMBOL(__ipipe_mach_set_dec); unsigned long __ipipe_mach_get_dec(void) { return OSMR0 - OSCR; } #endif /* CONFIG_IPIPE */ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-11 6:20 ` Detlef Vollmann @ 2006-07-11 15:01 ` Stelian Pop 2006-07-12 13:13 ` Bart Jonkers 2006-07-11 15:02 ` Stelian Pop 2006-07-17 16:17 ` Gilles Chanteperdrix 2 siblings, 1 reply; 36+ messages in thread From: Stelian Pop @ 2006-07-11 15:01 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai Le mardi 11 juillet 2006 à 08:20 +0200, Detlef Vollmann a écrit : > What is missing is a look at entry-macro.S. > Stelian Pop has done something for the Integrator that I don't > really understand and therefore I can't say whether the PXA needs > something similar. Well, you should have asked if you didn't undestand. :) The change in entry-macro.S does optimize the fast path for a timer interrupt. Instead of looking at each interrupt controller status and compute the irq number the code tests the timer interrupt status and returns immediately if true. Stelian. -- Stelian Pop <stelian.pop@domain.hid> ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-11 15:01 ` Stelian Pop @ 2006-07-12 13:13 ` Bart Jonkers 0 siblings, 0 replies; 36+ messages in thread From: Bart Jonkers @ 2006-07-12 13:13 UTC (permalink / raw) To: Stelian Pop; +Cc: xenomai On Tue, 2006-07-11 at 17:01 +0200, Stelian Pop wrote: > Le mardi 11 juillet 2006 à 08:20 +0200, Detlef Vollmann a écrit : > > > What is missing is a look at entry-macro.S. > > Stelian Pop has done something for the Integrator that I don't > > really understand and therefore I can't say whether the PXA needs > > something similar. > > Well, you should have asked if you didn't undestand. :) > > The change in entry-macro.S does optimize the fast path for a timer > interrupt. Instead of looking at each interrupt controller status and > compute the irq number the code tests the timer interrupt status and > returns immediately if true. So this isn't really needed? Are there other things where I have to take a look to? I think I have found 2 errors in the code of Detlef Vollmann: - timer_initialized is never set to 1, so __ipipe_mach_get_tsc will always return 0. I have set timer_initialized to 1 at the end of pxa_timer_init. - __ipipe_mach_set_dec will return without calling spin_unlock_irqrestore_hw(&timer_lock, flags); when delay < 8 When I use the adaptions of Detlef Vollmann and adeos-ipipe-2.6.15-arm-1.3-04 Linux doesn't get any interrupts. During initialization of the hard-disk I get the error 'hda: lost interrupt'. Somebody an idea? Bart > > Stelian. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-11 6:20 ` Detlef Vollmann 2006-07-11 15:01 ` Stelian Pop @ 2006-07-11 15:02 ` Stelian Pop 2006-07-17 16:17 ` Gilles Chanteperdrix 2 siblings, 0 replies; 36+ messages in thread From: Stelian Pop @ 2006-07-11 15:02 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai Le mardi 11 juillet 2006 à 08:20 +0200, Detlef Vollmann a écrit : > What is missing is a look at entry-macro.S. > Stelian Pop has done something for the Integrator that I don't > really understand and therefore I can't say whether the PXA needs > something similar. Well, you should have asked if you didn't undestand. :) The change in entry-macro.S does optimize the fast path for a timer interrupt. Instead of looking at each interrupt controller status and compute the irq number the code tests the timer interrupt status and returns immediately if true. Stelian. -- Stelian Pop <stelian.pop@domain.hid> ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-11 6:20 ` Detlef Vollmann 2006-07-11 15:01 ` Stelian Pop 2006-07-11 15:02 ` Stelian Pop @ 2006-07-17 16:17 ` Gilles Chanteperdrix 2006-07-17 16:29 ` Philippe Gerum 2006-07-17 22:33 ` Danilo Levantesi 2 siblings, 2 replies; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-07-17 16:17 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai [-- Attachment #1: message body and .signature --] [-- Type: text/plain, Size: 1605 bytes --] Detlef Vollmann wrote: > Gilles Chanteperdrix wrote: > > Danilo Levantesi wrote: > > > So what is the status of the port? Is anything working on a pxa arch? > > > > More recently, Detlef Vollmann seem also to have begun a port to > > PXA. > Unfortunately my priorities were shifted, so I'm not able to > work on it for the next few weeks. > > What I've done so far is attached. > IT'S UNTESTED!!! > So it's nothing you can just use, but something that you can build > on (possibly). > I'd be happy to hear about any experience with this. > > What is missing is a look at entry-macro.S. > Stelian Pop has done something for the Integrator that I don't > really understand and therefore I can't say whether the PXA needs > something similar. > > Detlef Hi, Starting from Detlef patch, I made some tests on an SA1100 based ARM, attached is a patch that works for me, which I also adapted to PXA without testing it. It would be nice if someone could test it on PXA. In order to do these tests, I had to adapt the ARM ipipe 2.6.15 patch to linux 2.6.16, so the 2.6.16 ipipe patch is attached too for further testing. Note that I had to make a small modification in include/asm-arm/system.h, because the SA1100 version of __xchg uses local_irq_restore, which use PSR_I_BIT, which is defined elsewhere, so I replaced PSR_I_BIT with its value. I now wonder if it is the proper fix, or if __xchg should be fixed to use local_irq_save_hw and local_irq_restore_hw instead, as is the case for the atomic operations defined in atomic.h and bitops.h. -- Gilles Chanteperdrix. [-- Attachment #2: ipipe-sa1100-pxa.patch --] [-- Type: text/plain, Size: 6770 bytes --] --- linux-2.6.16.5-tcl1/arch/arm/mach-pxa/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c 2006-07-17 17:54:09.000000000 +0200 @@ -30,6 +30,22 @@ #include <asm/arch/pxa-regs.h> +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int pxa_timer_initialized; +#endif /* CONFIG_IPIPE */ + static inline unsigned long pxa_get_rtc_time(void) { return RCNR; @@ -105,6 +121,19 @@ pxa_timer_interrupt(int irq, void *dev_i * affect things only when the timer IRQ has been delayed by nearly * exactly one tick period which should be a pretty rare event. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) + timer_tick(regs); + else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); OSSR = OSSR_M0; /* Clear match on timer 0 */ @@ -139,6 +168,10 @@ static void __init pxa_timer_init(void) setup_irq(IRQ_OST0, &pxa_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + pxa_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -216,3 +249,69 @@ struct sys_timer pxa_timer = { .dyn_tick = &pxa_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(sa1100_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + if (delay > 8) { + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); + } else + ipipe_trigger_irq(IRQ_OST0); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ --- linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c 2006-07-17 17:47:58.000000000 +0200 @@ -13,6 +13,7 @@ #include <linux/interrupt.h> #include <linux/timex.h> #include <linux/signal.h> +#include <linux/module.h> #include <asm/mach/time.h> #include <asm/hardware.h> @@ -20,6 +21,22 @@ #define RTC_DEF_DIVIDER (32768 - 1) #define RTC_DEF_TRIM 0 +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int sa1100_timer_initialized; +#endif /* CONFIG_IPIPE */ + static unsigned long __init sa1100_get_rtc_time(void) { /* @@ -97,6 +114,19 @@ sa1100_timer_interrupt(int irq, void *de * ensured, hence we can use do_gettimeofday() from interrupt * handlers. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) + timer_tick(regs); + else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); OSSR = OSSR_M0; /* Clear match on timer 0 */ @@ -131,6 +161,10 @@ static void __init sa1100_timer_init(voi setup_irq(IRQ_OST0, &sa1100_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + sa1100_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -209,3 +243,66 @@ struct sys_timer sa1100_timer = { .dyn_tick = &sa1100_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(sa1100_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ [-- Attachment #3: adeos-ipipe-2.6.16-arm-unofficial.patch --] [-- Type: text/plain, Size: 145698 bytes --] diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/Kconfig linux-2.6.16.5-tcl1-ipipe/arch/arm/Kconfig --- linux-2.6.16.5-tcl1/arch/arm/Kconfig 2006-07-15 20:06:02.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/Kconfig 2006-07-16 15:01:24.000000000 +0200 @@ -401,6 +401,8 @@ config LOCAL_TIMERS accounting to be spread across the timer interval, preventing a "thundering herd" at every timer tick. +source "kernel/ipipe/Kconfig" + config PREEMPT bool "Preemptible Kernel (EXPERIMENTAL)" depends on EXPERIMENTAL diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/Makefile linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/Makefile --- linux-2.6.16.5-tcl1/arch/arm/kernel/Makefile 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/Makefile 2006-07-16 15:19:02.000000000 +0200 @@ -21,6 +21,7 @@ obj-$(CONFIG_ISA_DMA) += dma-isa.o obj-$(CONFIG_PCI) += bios32.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_OABI_COMPAT) += sys_oabi-compat.o +obj-$(CONFIG_IPIPE) += ipipe-core.o ipipe-root.o obj-$(CONFIG_IWMMXT) += iwmmxt.o AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-armv.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-armv.S --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-armv.S 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-armv.S 2006-07-16 15:19:54.000000000 +0200 @@ -4,6 +4,7 @@ * Copyright (C) 1996,1997,1998 Russell King. * ARM700 fix by Matthew Godbolt (linux-user@domain.hid) * nommu support by Hyok S. Choi (hyok.choi@domain.hid) + * Copyright (C) 2005 Stelian Pop. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2 as @@ -33,7 +34,11 @@ @ routine called with r0 = irq number, r1 = struct pt_regs * @ adrne lr, 1b +#ifdef CONFIG_IPIPE + bne __ipipe_grab_irq +#else bne asm_do_IRQ +#endif #ifdef CONFIG_SMP /* @@ -199,6 +204,11 @@ __irq_svc: #endif irq_handler +#ifdef CONFIG_IPIPE + cmp r0, #0 + beq __ipipe_fast_svc_irq_exit +#endif + #ifdef CONFIG_PREEMPT ldr r0, [tsk, #TI_FLAGS] @ get flags tst r0, #_TIF_NEED_RESCHED @@ -209,6 +219,9 @@ preempt_return: teq r0, r7 strne r0, [r0, -r0] @ bug() #endif +#ifdef CONFIG_IPIPE +__ipipe_fast_svc_irq_exit: +#endif ldr r0, [sp, #S_PSR] @ irqs are already disabled msr spsr_cxsf, r0 ldmia sp, {r0 - pc}^ @ load r0 - pc, cpsr @@ -237,6 +250,12 @@ svc_preempt: __und_svc: svc_entry +#ifdef CONFIG_IPIPE + mov r1, sp @ r0 = trapno, r1 = ®s + bl __ipipe_dispatch_event @ branch to trap handler + cmp r0, #0 + bne 1f +#endif /* CONFIG_IPIPE */ @ @ call emulation code, which returns using r9 if it has emulated @ the instruction, or the more conventional lr if we are to treat @@ -406,6 +425,12 @@ __irq_usr: #endif irq_handler +#ifdef CONFIG_IPIPE + cmp r0, #0 + bne __ipipe_usr_irq_continue + slow_restore_user_regs @ Fast exit path over non-root domains +__ipipe_usr_irq_continue: +#endif #ifdef CONFIG_PREEMPT ldr r0, [tsk, #TI_PREEMPT] str r8, [tsk, #TI_PREEMPT] @@ -499,8 +524,8 @@ call_fpe: mov pc, lr @ CP#8 mov pc, lr @ CP#9 #ifdef CONFIG_VFP - b do_vfp @ CP#10 (VFP) - b do_vfp @ CP#11 (VFP) + b _do_vfp @ CP#10 (VFP) + b _do_vfp @ CP#11 (VFP) #else mov pc, lr @ CP#10 (VFP) mov pc, lr @ CP#11 (VFP) @@ -511,10 +536,34 @@ call_fpe: mov pc, lr @ CP#15 (Control) do_fpe: +#ifdef CONFIG_IPIPE + mov r4, r0 + mov r0, #5 @ == IPIPE_TRAP_FPU + mov r1, sp @ r0 = trapno, r1 = ®s + bl __ipipe_dispatch_event @ branch to trap handler + cmp r0, #0 + ldrne pc, [r9] + mov r0, r4 +#endif ldr r4, .LCfp add r10, r10, #TI_FPSTATE @ r10 = workspace ldr pc, [r4] @ Call FP module USR entry point +#ifdef CONFIG_VFP +_do_vfp: +#ifdef CONFIG_IPIPE + mov r4, r0 + mov r0, #6 @ == IPIPE_TRAP_VFP + mov r1, sp @ r0 = trapno, r1 = ®s + bl __ipipe_dispatch_event @ branch to trap handler + cmp r0, #0 + ldrne pc, [r9] + mov r0, r4 +#endif + b do_vfp +#endif + + /* * The FP module is called with these registers set: * r0 = instruction diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-common.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-common.S --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-common.S 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-common.S 2006-07-16 16:15:08.000000000 +0200 @@ -2,6 +2,7 @@ * linux/arch/arm/kernel/entry-common.S * * Copyright (C) 2000 Russell King + * Copyright (C) 2005 Stelian Pop. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2 as @@ -20,13 +21,18 @@ * possible here, and this includes saving r0 back into the SVC * stack. */ +#ifdef CONFIG_IPIPE +__ipipe_ret_fast_syscall: + ldr r0, [sp, #S_R0+S_OFF] @ returned r0 + /* fall through */ +#endif ret_fast_syscall: disable_irq @ disable interrupts ldr r1, [tsk, #TI_FLAGS] tst r1, #_TIF_WORK_MASK bne fast_work_pending - @ fast_restore_user_regs +fast_restore_user_regs: ldr r1, [sp, #S_OFF + S_PSR] @ get calling cpsr ldr lr, [sp, #S_OFF + S_PC]! @ get pc msr spsr_cxsf, r1 @ save in spsr_svc @@ -35,6 +41,13 @@ ret_fast_syscall: add sp, sp, #S_FRAME_SIZE - S_PC movs pc, lr @ return & move spsr_svc into cpsr +#ifdef CONFIG_IPIPE +__ipipe_fast_exit_syscall: + ldr r0, [sp, #S_R0+S_OFF] @ returned r0 + disable_irq @ disable interrupts + b fast_restore_user_regs +#endif /* CONFIG_IPIPE */ + /* * Ok, we need to do extra processing, enter the slow path. */ @@ -62,19 +75,15 @@ ret_slow_syscall: tst r1, #_TIF_WORK_MASK bne work_pending no_work_pending: - @ slow_restore_user_regs - ldr r1, [sp, #S_PSR] @ get calling cpsr - ldr lr, [sp, #S_PC]! @ get pc - msr spsr_cxsf, r1 @ save in spsr_svc - ldmdb sp, {r0 - lr}^ @ get calling r1 - lr - mov r0, r0 - add sp, sp, #S_FRAME_SIZE - S_PC - movs pc, lr @ return & move spsr_svc into cpsr + slow_restore_user_regs /* * This is how we return from a fork. */ ENTRY(ret_from_fork) +#ifdef CONFIG_IPIPE + enable_irq +#endif /* CONFIG_IPIPE */ bl schedule_tail get_thread_info tsk ldr r1, [tsk, #TI_FLAGS] @ check for syscall tracing @@ -197,8 +206,18 @@ ENTRY(vector_swi) bic scno, scno, #0xff000000 @ mask off SWI op-code eor scno, scno, #__NR_SYSCALL_BASE @ check OS number #endif - stmdb sp!, {r4, r5} @ push fifth and sixth args +#ifdef CONFIG_IPIPE + stmfd sp!, {r0-r3, ip} + add r1, sp, #S_OFF + add r1, r1, #20 + mov r0, scno + bl __ipipe_syscall_root + cmp r0, #0 + ldmfd sp!, {r0-r3, ip} + blt __ipipe_ret_fast_syscall + bgt __ipipe_fast_exit_syscall +#endif /* CONFIG_IPIPE */ tst ip, #_TIF_SYSCALL_TRACE @ are we tracing syscalls? bne __sys_trace @@ -245,6 +264,9 @@ __sys_trace_return: __cr_alignment: .word cr_alignment #endif +#ifdef CONFIG_IPIPE + .word __ipipe_syscall_root +#endif .ltorg /* diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-header.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-header.S --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-header.S 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-header.S 2006-07-16 15:01:24.000000000 +0200 @@ -68,6 +68,15 @@ #endif .endm + .macro slow_restore_user_regs + ldr r1, [sp, #S_PSR] @ get calling cpsr + ldr lr, [sp, #S_PC]! @ get pc + msr spsr_cxsf, r1 @ save in spsr_svc + ldmdb sp, {r0 - lr}^ @ get calling r1 - lr + mov r0, r0 + add sp, sp, #S_FRAME_SIZE - S_PC + movs pc, lr @ return & move spsr_svc into cpsr + .endm /* * These are the registers used in the syscall handler, and allow us to diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-core.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-core.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-core.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-core.c 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,239 @@ +/* -*- linux-c -*- + * linux/arch/arm/kernel/ipipe-core.c + * + * Copyright (C) 2002-2005 Philippe Gerum. + * Copyright (C) 2004 Wolfgang Grandegger (Adeos/arm port over 2.4). + * Copyright (C) 2005 Heikki Lindholm (PowerPC 970 fixes). + * Copyright (C) 2005 Stelian Pop. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-dependent I-PIPE core support for ARM. + */ + +#include <linux/config.h> +#include <linux/kernel.h> +#include <linux/smp.h> +#include <linux/sched.h> +#include <linux/slab.h> +#include <linux/bitops.h> +#include <linux/interrupt.h> +#include <linux/module.h> +#include <asm/system.h> +#include <asm/atomic.h> +#include <asm/hardirq.h> +#include <asm/io.h> + +/* Current reload value for the decrementer. */ +unsigned long __ipipe_decr_ticks; + +/* Next tick date (timebase value). */ +unsigned long long __ipipe_decr_next[IPIPE_NR_CPUS]; + +struct pt_regs __ipipe_tick_regs[IPIPE_NR_CPUS]; + +#ifdef CONFIG_SMP + +static cpumask_t __ipipe_cpu_sync_map; + +static cpumask_t __ipipe_cpu_lock_map; + +static ipipe_spinlock_t __ipipe_cpu_barrier = IPIPE_SPIN_LOCK_UNLOCKED; + +static atomic_t __ipipe_critical_count = ATOMIC_INIT(0); + +static void (*__ipipe_cpu_sync) (void); + +/* Always called with hw interrupts off. */ + +void __ipipe_do_critical_sync(unsigned irq) +{ + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + + cpu_set(cpuid, __ipipe_cpu_sync_map); + + /* + * Now we are in sync with the lock requestor running on another + * CPU. Enter a spinning wait until he releases the global + * lock. + */ + spin_lock_hw(&__ipipe_cpu_barrier); + + /* Got it. Now get out. */ + + if (__ipipe_cpu_sync) + /* Call the sync routine if any. */ + __ipipe_cpu_sync(); + + spin_unlock_hw(&__ipipe_cpu_barrier); + + cpu_clear(cpuid, __ipipe_cpu_sync_map); +} + +#endif /* CONFIG_SMP */ + +/* + * ipipe_critical_enter() -- Grab the superlock excluding all CPUs + * but the current one from a critical section. This lock is used when + * we must enforce a global critical section for a single CPU in a + * possibly SMP system whichever context the CPUs are running. + */ +unsigned long ipipe_critical_enter(void (*syncfn) (void)) +{ + unsigned long flags; + + local_irq_save_hw(flags); + +#ifdef CONFIG_SMP + if (num_online_cpus() > 1) { /* We might be running a SMP-kernel on a UP box... */ + ipipe_declare_cpuid; + cpumask_t lock_map; + + ipipe_load_cpuid(); + + if (!cpu_test_and_set(cpuid, __ipipe_cpu_lock_map)) { + while (cpu_test_and_set(BITS_PER_LONG - 1, + __ipipe_cpu_lock_map)) { + int n = 0; + do { + cpu_relax(); + } while (++n < cpuid); + } + + spin_lock_hw(&__ipipe_cpu_barrier); + + __ipipe_cpu_sync = syncfn; + + /* Send the sync IPI to all processors but the current one. */ + send_IPI_allbutself(IPIPE_CRITICAL_VECTOR); + + cpus_andnot(lock_map, cpu_online_map, + __ipipe_cpu_lock_map); + + while (!cpus_equal(__ipipe_cpu_sync_map, lock_map)) + cpu_relax(); + } + + atomic_inc(&__ipipe_critical_count); + } +#endif /* CONFIG_SMP */ + + return flags; +} + +/* ipipe_critical_exit() -- Release the superlock. */ + +void ipipe_critical_exit(unsigned long flags) +{ +#ifdef CONFIG_SMP + if (num_online_cpus() > 1) { /* We might be running a SMP-kernel on a UP box... */ + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + + if (atomic_dec_and_test(&__ipipe_critical_count)) { + spin_unlock_hw(&__ipipe_cpu_barrier); + + while (!cpus_empty(__ipipe_cpu_sync_map)) + cpu_relax(); + + cpu_clear(cpuid, __ipipe_cpu_lock_map); + cpu_clear(BITS_PER_LONG - 1, __ipipe_cpu_lock_map); + } + } +#endif /* CONFIG_SMP */ + + local_irq_restore_hw(flags); +} + +void __ipipe_init_platform(void) +{ + __ipipe_decr_ticks = __ipipe_mach_ticks_per_jiffy; +} + +int ipipe_get_sysinfo(struct ipipe_sysinfo *info) +{ + info->ncpus = num_online_cpus(); + info->cpufreq = ipipe_cpu_freq(); + info->archdep.tmirq = __ipipe_mach_timerint; + info->archdep.tmfreq = info->cpufreq; + + return 0; +} + +/* + * ipipe_trigger_irq() -- Push the interrupt at front of the pipeline + * just like if it has been actually received from a hw source. Also + * works for virtual interrupts. + */ +int ipipe_trigger_irq(unsigned irq) +{ + unsigned long flags; + + if (irq >= IPIPE_NR_IRQS || + (ipipe_virtual_irq_p(irq) + && !test_bit(irq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map))) + return -EINVAL; + + local_irq_save_hw(flags); + + __ipipe_handle_irq(irq, NULL); + + local_irq_restore_hw(flags); + + return 1; +} + +static void __ipipe_set_decr(void) +{ + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + + __ipipe_decr_next[cpuid] = __ipipe_read_timebase() + __ipipe_decr_ticks; + __ipipe_mach_set_dec(__ipipe_decr_ticks); +} + +int ipipe_tune_timer(unsigned long ns, int flags) +{ + unsigned long x, ticks; + + if (flags & IPIPE_RESET_TIMER) + ticks = __ipipe_mach_ticks_per_jiffy; + else { + ticks = (ns / 1000) * (__ipipe_mach_ticks_per_jiffy) / (1000000 / HZ); + + if (ticks > __ipipe_mach_ticks_per_jiffy) + return -EINVAL; + } + + x = ipipe_critical_enter(&__ipipe_set_decr); /* Sync with all CPUs */ + __ipipe_decr_ticks = ticks; + __ipipe_set_decr(); + ipipe_critical_exit(x); + + return 0; +} + +EXPORT_SYMBOL(__ipipe_decr_ticks); +EXPORT_SYMBOL(__ipipe_decr_next); +EXPORT_SYMBOL(ipipe_critical_enter); +EXPORT_SYMBOL(ipipe_critical_exit); +EXPORT_SYMBOL(ipipe_trigger_irq); +EXPORT_SYMBOL(ipipe_get_sysinfo); +EXPORT_SYMBOL(ipipe_tune_timer); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-root.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-root.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c 2006-07-17 14:59:06.000000000 +0200 @@ -0,0 +1,379 @@ +/* -*- linux-c -*- + * linux/arch/arm/kernel/ipipe-root.c + * + * Copyright (C) 2002-2005 Philippe Gerum (Adeos/ppc port over 2.6). + * Copyright (C) 2004 Wolfgang Grandegger (Adeos/ppc port over 2.4). + * Copyright (C) 2005 Stelian Pop. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-dependent I-pipe support for ARM. + */ + +#include <linux/config.h> +#include <linux/kernel.h> +#include <linux/sched.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/interrupt.h> +#include <linux/errno.h> +#include <asm/system.h> +#include <asm/hardirq.h> +#include <asm/atomic.h> +#include <asm/io.h> +#include <asm/irq.h> +#include <asm/unistd.h> +#include <asm/mach/irq.h> +#include <asm/mmu_context.h> + +extern struct irqdesc irq_desc[]; +extern spinlock_t irq_controller_lock; +extern asmlinkage void asm_do_IRQ(unsigned int irq, struct pt_regs *regs); + +static struct irqchip __ipipe_std_irq_dtype[NR_IRQS]; + +static void __ipipe_override_irq_unmask(unsigned irq) +{ + unsigned long flags; + + local_irq_save_hw(flags); + ipipe_irq_unlock(irq); + __ipipe_std_irq_dtype[irq].unmask(irq); + local_irq_restore_hw(flags); +} + +static void __ipipe_override_irq_mask(unsigned irq) +{ + unsigned long flags; + + local_irq_save_hw(flags); + ipipe_irq_lock(irq); + __ipipe_std_irq_dtype[irq].mask(irq); + local_irq_restore_hw(flags); +} + +static void __ipipe_override_irq_mask_ack(unsigned irq) +{ + unsigned long flags; + + local_irq_save_hw(flags); + ipipe_irq_lock(irq); + __ipipe_std_irq_dtype[irq].ack(irq); + local_irq_restore_hw(flags); +} + + +static void __ipipe_enable_sync(void) +{ + __ipipe_decr_next[ipipe_processor_id()] = + __ipipe_read_timebase() + __ipipe_mach_get_dec(); +} + +/* + * __ipipe_enable_pipeline() -- We are running on the boot CPU, hw + * interrupts are off, and secondary CPUs are still lost in space. + */ +void __ipipe_enable_pipeline(void) +{ + unsigned long flags; + unsigned irq; + + flags = ipipe_critical_enter(&__ipipe_enable_sync); + + /* First, virtualize all interrupts from the root domain. */ + + for (irq = 0; irq < NR_IRQS; irq++) + ipipe_virtualize_irq(ipipe_root_domain, + irq, + (ipipe_irq_handler_t)&asm_do_IRQ, NULL, + (irq == __ipipe_mach_timerint) ? &__ipipe_ack_timerirq : &__ipipe_ack_irq, + IPIPE_HANDLE_MASK | IPIPE_PASS_MASK); + + /* + * Interpose on the IRQ control routines so we can make them + * atomic using hw masking and prevent the interrupt log from + * being untimely flushed. + */ + + for (irq = 0; irq < NR_IRQS; irq++) + __ipipe_std_irq_dtype[irq] = *irq_desc[irq].chip; + + /* + * The original controller structs are often shared, so we first + * save them all before changing any of them. Notice that we don't + * override the ack() handler since we will enforce the necessary + * setup in __ipipe_ack_irq(). + */ + + for (irq = 0; irq < NR_IRQS; irq++) { + if (irq_desc[irq].chip->mask != NULL) + irq_desc[irq].chip->mask = __ipipe_override_irq_mask; + + if (irq_desc[irq].chip->unmask != NULL) + irq_desc[irq].chip->unmask = __ipipe_override_irq_unmask; + + if (irq_desc[irq].chip->ack != NULL) + irq_desc[irq].chip->ack = __ipipe_override_irq_mask_ack; + } + + __ipipe_decr_next[ipipe_processor_id()] = + __ipipe_read_timebase() + __ipipe_mach_get_dec(); + + ipipe_critical_exit(flags); +} + +int __ipipe_ack_irq(unsigned irq) +{ + unsigned long flags; + ipipe_declare_cpuid; + + /* + * No need to mask IRQs at hw level: we are always called from + * __ipipe_handle_irq(), so interrupts are already off. We + * stall the pipeline so that spin_lock_irq*() ops won't + * unintentionally flush it, since this could cause infinite + * recursion. + */ + + ipipe_load_cpuid(); + flags = ipipe_test_and_stall_pipeline(); + preempt_disable(); + spin_lock_hw(&irq_controller_lock); + __ipipe_std_irq_dtype[irq].ack(irq); + spin_unlock_hw(&irq_controller_lock); + preempt_enable_no_resched(); + ipipe_restore_pipeline_nosync(ipipe_percpu_domain[cpuid], flags, cpuid); + + return 1; +} + +int __ipipe_ack_timerirq(unsigned irq) +{ + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + flags = ipipe_test_and_stall_pipeline(); + preempt_disable(); + spin_lock_hw(&irq_controller_lock); + __ipipe_mach_acktimer(); + __ipipe_std_irq_dtype[irq].ack(irq); + __ipipe_std_irq_dtype[irq].unmask(irq); + spin_unlock_hw(&irq_controller_lock); + preempt_enable_no_resched(); + ipipe_restore_pipeline_nosync(ipipe_percpu_domain[cpuid], flags, cpuid); + + return 1; +} + +/* + * __ipipe_handle_irq() -- IPIPE's generic IRQ handler. An optimistic + * interrupt protection log is maintained here for each domain. Hw + * interrupts are off on entry. + */ +void __ipipe_handle_irq(int irq, struct pt_regs *regs) +{ + struct ipipe_domain *this_domain, *next_domain; + struct list_head *head, *pos; + ipipe_declare_cpuid; + int m_ack, s_ack; + + m_ack = (regs == NULL); + + if (irq >= IPIPE_NR_IRQS) { + printk(KERN_ERR "I-pipe: spurious interrupt %d\n", irq); + return; + } + + ipipe_load_cpuid(); + + this_domain = ipipe_percpu_domain[cpuid]; + + if (test_bit(IPIPE_STICKY_FLAG, &this_domain->irqs[irq].control)) + head = &this_domain->p_link; + else { + head = __ipipe_pipeline.next; + next_domain = list_entry(head, struct ipipe_domain, p_link); + if (likely(test_bit(IPIPE_WIRED_FLAG, &next_domain->irqs[irq].control))) { + if (!m_ack && next_domain->irqs[irq].acknowledge != NULL) + next_domain->irqs[irq].acknowledge(irq); + if (likely(__ipipe_dispatch_wired(next_domain, irq))) + goto finalize; + return; + } + } + + /* Ack the interrupt. */ + + s_ack = m_ack; + pos = head; + + while (pos != &__ipipe_pipeline) { + next_domain = list_entry(pos, struct ipipe_domain, p_link); + + /* + * For each domain handling the incoming IRQ, mark it as + * pending in its log. + */ + if (test_bit(IPIPE_HANDLE_FLAG, + &next_domain->irqs[irq].control)) { + /* + * Domains that handle this IRQ are polled for + * acknowledging it by decreasing priority order. The + * interrupt must be made pending _first_ in the + * domain's status flags before the PIC is unlocked. + */ + + next_domain->cpudata[cpuid].irq_counters[irq].total_hits++; + next_domain->cpudata[cpuid].irq_counters[irq].pending_hits++; + __ipipe_set_irq_bit(next_domain, cpuid, irq); + + /* + * Always get the first master acknowledge available. + * Once we've got it, allow slave acknowledge + * handlers to run (until one of them stops us). + */ + if (next_domain->irqs[irq].acknowledge != NULL) { + if (!m_ack) + m_ack = next_domain->irqs[irq].acknowledge(irq); + else if (test_bit + (IPIPE_SHARED_FLAG, + &next_domain->irqs[irq].control) && !s_ack) + s_ack = next_domain->irqs[irq].acknowledge(irq); + } + } + + /* + * If the domain does not want the IRQ to be passed down the + * interrupt pipe, exit the loop now. + */ + + if (!test_bit(IPIPE_PASS_FLAG, &next_domain->irqs[irq].control)) + break; + + pos = next_domain->p_link.next; + } + +finalize: + /* + * Now walk the pipeline, yielding control to the highest + * priority domain that has pending interrupt(s) or + * immediately to the current domain if the interrupt has been + * marked as 'sticky'. This search does not go beyond the + * current domain in the pipeline. + */ + + __ipipe_walk_pipeline(head, cpuid); +} + +asmlinkage int __ipipe_grab_irq(int irq, struct pt_regs *regs) +{ + ipipe_declare_cpuid; + + if (irq == __ipipe_mach_timerint) { + + __ipipe_tick_regs[cpuid].ARM_cpsr = regs->ARM_cpsr; + __ipipe_tick_regs[cpuid].ARM_pc = regs->ARM_pc; + + if (__ipipe_decr_ticks != __ipipe_mach_ticks_per_jiffy) { + unsigned long long next_date, now; + + next_date = __ipipe_decr_next[cpuid]; + + while ((now = __ipipe_read_timebase()) >= next_date) + next_date += __ipipe_decr_ticks; + + __ipipe_mach_set_dec(next_date - now); + + __ipipe_decr_next[cpuid] = next_date; + } + } + + __ipipe_handle_irq(irq, regs); + + ipipe_load_cpuid(); + + return (ipipe_percpu_domain[cpuid] == ipipe_root_domain && + !test_bit(IPIPE_STALL_FLAG, + &ipipe_root_domain->cpudata[cpuid].status)); +} + +asmlinkage int __ipipe_check_root(struct pt_regs *regs) +{ + ipipe_declare_cpuid; + /* + * This routine is called with hw interrupts off, so no migration + * can occur while checking the identity of the current domain. + */ + ipipe_load_cpuid(); + return (ipipe_percpu_domain[cpuid] == ipipe_root_domain && + !test_bit(IPIPE_STALL_FLAG, + &ipipe_root_domain->cpudata[cpuid].status)); +} + +asmlinkage int __ipipe_syscall_root(unsigned long scno, struct pt_regs *regs) +{ + ipipe_declare_cpuid; + unsigned long flags, origr7; + + /* We use r7 to pass the syscall number to the other domains */ + origr7 = regs->ARM_r7; + regs->ARM_r7 = __NR_SYSCALL_BASE + scno; + + /* + * This routine either returns: + * 0 -- if the syscall is to be passed to Linux; + * >0 -- if the syscall should not be passed to Linux, and no + * tail work should be performed; + * <0 -- if the syscall should not be passed to Linux but the + * tail work has to be performed (for handling signals etc). + */ + + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SYSCALL) && + __ipipe_dispatch_event(IPIPE_EVENT_SYSCALL,regs) > 0) { + /* + * We might enter here over a non-root domain and exit + * over the root one as a result of the syscall + * (i.e. by recycling the register set of the current + * context across the migration), so we need to fixup + * the interrupt flag upon return too, so that + * __ipipe_unstall_iret_root() resets the correct + * stall bit on exit. + */ + if (ipipe_current_domain == ipipe_root_domain && !in_atomic()) { + /* + * Sync pending VIRQs before _TIF_NEED_RESCHED + * is tested. + */ + ipipe_lock_cpu(flags); + if ((ipipe_root_domain->cpudata[cpuid].irq_pending_hi & IPIPE_IRQMASK_VIRT) != 0) + __ipipe_sync_pipeline(IPIPE_IRQMASK_VIRT); + ipipe_unlock_cpu(flags); + regs->ARM_r7 = origr7; + return -1; + } + regs->ARM_r7 = origr7; + return 1; + } + + regs->ARM_r7 = origr7; + return 0; +} + +EXPORT_SYMBOL_GPL(show_stack); +#ifndef MULTI_CPU +EXPORT_SYMBOL_GPL(cpu_do_switch_mm); +#endif /* MULTI_CPU */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/irq.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/irq.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/irq.c 2006-07-16 15:01:24.000000000 +0200 @@ -54,10 +54,11 @@ static int noirqdebug; static volatile unsigned long irq_err_count; -static DEFINE_SPINLOCK(irq_controller_lock); +DEFINE_SPINLOCK(irq_controller_lock); static LIST_HEAD(irq_pending); struct irqdesc irq_desc[NR_IRQS]; +EXPORT_SYMBOL(irq_desc); void (*init_arch_irq)(void) __initdata = NULL; /* @@ -412,7 +413,9 @@ do_edge_IRQ(unsigned int irq, struct irq /* * Acknowledge and clear the IRQ, but don't mask it. */ +#ifndef CONFIG_IPIPE desc->chip->ack(irq); +#endif /* CONFIG_IPIPE */ /* * Mark the IRQ currently in progress. @@ -450,8 +453,10 @@ do_edge_IRQ(unsigned int irq, struct irq * currently running. Delay it. */ desc->pending = 1; +#ifndef CONFIG_IPIPE desc->chip->mask(irq); desc->chip->ack(irq); +#endif /* CONFIG_IPIPE */ } /* @@ -468,7 +473,9 @@ do_level_IRQ(unsigned int irq, struct ir /* * Acknowledge, clear _AND_ disable the interrupt. */ +#ifndef CONFIG_IPIPE desc->chip->ack(irq); +#endif /* CONFIG_IPIPE */ if (likely(!desc->disable_depth)) { kstat_cpu(cpu).irqs[irq]++; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/process.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/process.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/process.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/process.c 2006-07-16 15:01:24.000000000 +0200 @@ -89,12 +89,12 @@ static void default_idle(void) if (hlt_counter) cpu_relax(); else { - local_irq_disable(); + local_irq_disable_hw(); if (!need_resched()) { timer_dyn_reprogram(); arch_idle(); } - local_irq_enable(); + local_irq_enable_hw(); } } @@ -120,6 +120,7 @@ void cpu_idle(void) if (!idle) idle = default_idle; + ipipe_suspend_domain(); leds_event(led_idle_start); while (!need_resched()) idle(); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ptrace.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ptrace.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/ptrace.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ptrace.c 2006-07-16 15:01:24.000000000 +0200 @@ -486,6 +486,10 @@ void ptrace_break(struct task_struct *ts static int break_trap(struct pt_regs *regs, unsigned int instr) { + + if (ipipe_trap_notify(IPIPE_TRAP_BREAK,regs)) + return 0; + ptrace_break(current, regs); return 0; } diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/traps.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/traps.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/traps.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/traps.c 2006-07-16 15:01:24.000000000 +0200 @@ -310,6 +310,9 @@ asmlinkage void do_undefinstr(struct pt_ } spin_unlock_irq(&undef_lock); + if (ipipe_trap_notify(IPIPE_TRAP_UNDEFINSTR,regs)) + return; + #ifdef CONFIG_DEBUG_USER if (user_debug & UDBG_UNDEFINED) { printk(KERN_INFO "%s (%d): undefined instruction: pc=%p\n", diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-integrator/core.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/core.c --- linux-2.6.16.5-tcl1/arch/arm/mach-integrator/core.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/core.c 2006-07-16 15:01:24.000000000 +0200 @@ -2,6 +2,7 @@ * linux/arch/arm/mach-integrator/core.c * * Copyright (C) 2000-2003 Deep Blue Solutions Ltd + * Copyright (C) 2005 Stelian Pop. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2, as @@ -148,53 +149,57 @@ EXPORT_SYMBOL(cm_control); /* * How long is the timer interval? */ -#define TIMER_INTERVAL (TICKS_PER_uSEC * mSEC_10) -#if TIMER_INTERVAL >= 0x100000 -#define TICKS2USECS(x) (256 * (x) / TICKS_PER_uSEC) -#elif TIMER_INTERVAL >= 0x10000 -#define TICKS2USECS(x) (16 * (x) / TICKS_PER_uSEC) -#else #define TICKS2USECS(x) ((x) / TICKS_PER_uSEC) -#endif static unsigned long timer_reload; +static unsigned long timer_interval; +static unsigned long timer_lxlost; +static int tscok; + +#ifdef CONFIG_IPIPE +int __ipipe_mach_timerint = IRQ_TIMERINT1; +static unsigned long long __ipipe_mach_tsc; +static DEFINE_SPINLOCK(timer_lock); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); +#endif /* - * Returns number of ms since last clock interrupt. Note that interrupts - * will have been disabled by do_gettimeoffset() + * Called with IRQ disabled from do_gettimeofday(). */ -unsigned long integrator_gettimeoffset(void) +static inline unsigned long integrator_getticksoffset(void) { - unsigned long ticks1, ticks2, status; + unsigned long ticks; - /* - * Get the current number of ticks. Note that there is a race - * condition between us reading the timer and checking for - * an interrupt. We get around this by ensuring that the - * counter has not reloaded between our two reads. - */ - ticks2 = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; - do { - ticks1 = ticks2; - status = __raw_readl(VA_IC_BASE + IRQ_RAW_STATUS); - ticks2 = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; - } while (ticks2 > ticks1); + if (!tscok) + return 0; - /* - * Number of ticks since last interrupt. - */ - ticks1 = timer_reload - ticks2; + ticks = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; - /* - * Interrupt pending? If so, we've reloaded once already. - */ - if (status & (1 << IRQ_TIMERINT1)) - ticks1 += timer_reload; + if (ticks > timer_reload) + ticks = 0xffff + timer_reload - ticks; + else + ticks = timer_reload - ticks; + if (timer_interval < 0x10000) + return ticks; + else if (timer_interval < 0x100000) + return ticks * 16; + else + return ticks * 256; +} + +/* + * Returns number of ms since last clock interrupt. Note that interrupts + * will have been disabled by do_gettimeoffset() + */ +unsigned long integrator_gettimeoffset(void) +{ /* * Convert the ticks to usecs */ - return TICKS2USECS(ticks1); + return TICKS2USECS(timer_lxlost + integrator_getticksoffset()); } /* @@ -205,10 +210,22 @@ integrator_timer_interrupt(int irq, void { write_seqlock(&xtime_lock); + timer_lxlost = 0; + +#ifdef CONFIG_IPIPE /* - * clear the interrupt + * If Linux is the only domain, ack the timer and reprogram it */ - writel(1, TIMER1_VA_BASE + TIMER_INTCLR); + if (!__ipipe_mach_timerstolen) { + __ipipe_mach_tsc += integrator_getticksoffset(); +#else + writel(1, TIMER1_VA_BASE + TIMER_INTCLR); +#endif + + writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); +#ifdef CONFIG_IPIPE + } +#endif /* * the clock tick routines are only processed on the @@ -239,24 +256,30 @@ static struct irqaction integrator_timer .handler = integrator_timer_interrupt, }; -/* - * Set up timer interrupt, and return the current time in seconds. - */ -void __init integrator_time_init(unsigned long reload, unsigned int ctrl) +static inline void set_dec(unsigned long reload) { - unsigned int timer_ctrl = TIMER_CTRL_ENABLE | TIMER_CTRL_PERIODIC; + unsigned int ctrl = TIMER_CTRL_ENABLE | TIMER_CTRL_IE; timer_reload = reload; - timer_ctrl |= ctrl; + timer_interval = reload; - if (timer_reload > 0x100000) { + if (timer_reload >= 0x100000) { timer_reload >>= 8; - timer_ctrl |= TIMER_CTRL_DIV256; - } else if (timer_reload > 0x010000) { + ctrl |= TIMER_CTRL_DIV256; + } else if (timer_reload >= 0x010000) { timer_reload >>= 4; - timer_ctrl |= TIMER_CTRL_DIV16; + ctrl |= TIMER_CTRL_DIV16; } + writel(ctrl, TIMER1_VA_BASE + TIMER_CTRL); + writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); +} + +/* + * Set up timer interrupt, and return the current time in seconds. + */ +void __init integrator_time_init(unsigned long reload, unsigned int ctrl) +{ /* * Initialise to a known state (all timers off) */ @@ -264,12 +287,51 @@ void __init integrator_time_init(unsigne writel(0, TIMER1_VA_BASE + TIMER_CTRL); writel(0, TIMER2_VA_BASE + TIMER_CTRL); - writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); - writel(timer_reload, TIMER1_VA_BASE + TIMER_VALUE); - writel(timer_ctrl, TIMER1_VA_BASE + TIMER_CTRL); + set_dec(reload); /* * Make irqs happen for the system timer */ setup_irq(IRQ_TIMERINT1, &integrator_timer_irq); + + tscok = 1; } + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + writel(1, TIMER1_VA_BASE + TIMER_INTCLR); +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + unsigned long long result; + unsigned long flags; + + spin_lock_irqsave_hw(&timer_lock, flags); + result = __ipipe_mach_tsc + integrator_getticksoffset(); + spin_unlock_irqrestore_hw(&timer_lock, flags); + return result; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +void __ipipe_mach_set_dec(unsigned long reload) +{ + unsigned long ticks; + unsigned long flags; + + spin_lock_irqsave_hw(&timer_lock, flags); + ticks = integrator_getticksoffset(); + __ipipe_mach_tsc += ticks; + timer_lxlost += ticks; + + set_dec(reload); + spin_unlock_irqrestore_hw(&timer_lock, flags); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return readl(TIMER1_VA_BASE + TIMER_VALUE); +} +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-integrator/integrator_cp.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c --- linux-2.6.16.5-tcl1/arch/arm/mach-integrator/integrator_cp.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c 2006-07-16 15:01:24.000000000 +0200 @@ -2,6 +2,7 @@ * linux/arch/arm/mach-integrator/integrator_cp.c * * Copyright (C) 2003 Deep Blue Solutions Ltd + * Copyright (C) 2005 Stelian Pop. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -568,9 +569,14 @@ static void __init intcp_init(void) #define TIMER_CTRL_IE (1 << 5) /* Interrupt Enable */ +#ifdef CONFIG_IPIPE +unsigned int __ipipe_mach_ticks_per_jiffy = 1000000 * TICKS_PER_uSEC / HZ; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); +#endif + static void __init intcp_timer_init(void) { - integrator_time_init(1000000 / HZ, TIMER_CTRL_IE); + integrator_time_init(1000000 * TICKS_PER_uSEC / HZ, TIMER_CTRL_IE); } static struct sys_timer cp_timer = { diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-pxa/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mm/fault.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mm/fault.c --- linux-2.6.16.5-tcl1/arch/arm/mm/fault.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mm/fault.c 2006-07-16 15:01:24.000000000 +0200 @@ -223,6 +223,9 @@ do_page_fault(unsigned long addr, unsign struct mm_struct *mm; int fault, sig, code; + if (ipipe_trap_notify(IPIPE_TRAP_ACCESS,regs)) + return 0; + tsk = current; mm = tsk->mm; @@ -354,6 +357,9 @@ do_translation_fault(unsigned long addr, bad_area: tsk = current; + if (ipipe_trap_notify(IPIPE_TRAP_ACCESS,regs)) + return 0; + do_bad_area(tsk, tsk->active_mm, addr, fsr, regs); return 0; } @@ -366,6 +372,10 @@ static int do_sect_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) { struct task_struct *tsk = current; + + if (ipipe_trap_notify(IPIPE_TRAP_SECTION,regs)) + return 0; + do_bad_area(tsk, tsk->active_mm, addr, fsr, regs); return 0; } @@ -376,6 +386,9 @@ do_sect_fault(unsigned long addr, unsign static int do_bad(unsigned long addr, unsigned int fsr, struct pt_regs *regs) { + if (ipipe_trap_notify(IPIPE_TRAP_DABT,regs)) + return 0; + return 1; } @@ -451,6 +464,9 @@ do_DataAbort(unsigned long addr, unsigne if (!inf->fn(addr, fsr, regs)) return; + if (ipipe_trap_notify(IPIPE_TRAP_UNKNOWN,regs)) + return; + printk(KERN_ALERT "Unhandled fault: %s (0x%03x) at 0x%08lx\n", inf->name, fsr, addr); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/drivers/pci/msi.c linux-2.6.16.5-tcl1-ipipe/drivers/pci/msi.c --- linux-2.6.16.5-tcl1/drivers/pci/msi.c 2006-05-07 16:41:40.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/drivers/pci/msi.c 2006-07-16 15:01:24.000000000 +0200 @@ -151,6 +151,21 @@ static void unmask_MSI_irq(unsigned int msi_set_mask_bit(vector, 0); } +#ifdef CONFIG_IPIPE +static void ack_MSI_irq_w_maskbit(unsigned int vector) +{ + mask_MSI_irq(vector); + __ack_APIC_irq(); +} +static void ack_MSI_irq_wo_maskbit(unsigned int vector) +{ + __ack_APIC_irq(); +} +#else /* !CONFIG_IPIPE */ +#define ack_MSI_irq_wo_maskbit do_nothing +#define ack_MSI_irq_w_maskbit mask_MSI_irq +#endif /* CONFIG_IPIPE */ + static unsigned int startup_msi_irq_wo_maskbit(unsigned int vector) { struct msi_desc *entry; @@ -214,7 +229,7 @@ static struct hw_interrupt_type msix_irq .shutdown = shutdown_msi_irq, .enable = unmask_MSI_irq, .disable = mask_MSI_irq, - .ack = mask_MSI_irq, + .ack = ack_MSI_irq_w_maskbit, .end = end_msi_irq_w_maskbit, .set_affinity = set_msi_affinity }; @@ -230,7 +245,7 @@ static struct hw_interrupt_type msi_irq_ .shutdown = shutdown_msi_irq, .enable = unmask_MSI_irq, .disable = mask_MSI_irq, - .ack = mask_MSI_irq, + .ack = ack_MSI_irq_w_maskbit, .end = end_msi_irq_w_maskbit, .set_affinity = set_msi_affinity }; @@ -246,7 +261,7 @@ static struct hw_interrupt_type msi_irq_ .shutdown = shutdown_msi_irq, .enable = do_nothing, .disable = do_nothing, - .ack = do_nothing, + .ack = ack_MSI_irq_wo_maskbit, .end = end_msi_irq_wo_maskbit, .set_affinity = set_msi_affinity }; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/entry-macro.S linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/entry-macro.S --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/entry-macro.S 2006-05-07 16:42:04.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/entry-macro.S 2006-07-16 15:01:24.000000000 +0200 @@ -22,7 +22,11 @@ teq \irqstat, #0 ldreq \irqstat, [\base, #(INTEGRATOR_HDR_IC_OFFSET+IRQ_STATUS)] moveq \irqnr, #IRQ_CIC_START - +#ifdef CONFIG_IPIPE + tst \irqstat, #0x00000040 @ check IRQ_TIMERINT1 first + movne \irqnr, #6 + bne 1003f +#endif /* CONFIG_IPIPE */ 1001: tst \irqstat, #15 bne 1002f add \irqnr, \irqnr, #4 diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/platform.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/platform.h --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/platform.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/platform.h 2006-07-16 15:01:24.000000000 +0200 @@ -26,13 +26,15 @@ * NOTE: This is a multi-hosted header file for use with uHAL and * supported debuggers. * - * $Id: platform.s,v 1.32 2000/02/18 10:51:39 asims Exp $ + * $Id: platform.h,v 1.2 2006/02/20 13:54:22 rpm Exp $ * * ***********************************************************************/ #ifndef __address_h #define __address_h 1 +#include <linux/config.h> + /* ======================================================================== * Integrator definitions * ======================================================================== @@ -436,7 +438,7 @@ * Timer definitions * * Only use timer 1 & 2 - * (both run at 24MHz and will need the clock divider set to 16). + * (both run at 1MHZ on /CP and at 24MHz on /AP) * * Timer 0 runs at bus frequency and therefore could vary and currently * uHAL can't handle that. @@ -449,7 +451,12 @@ #define MAX_TIMER 2 #define MAX_PERIOD 699050 + +#ifdef CONFIG_ARCH_INTEGRATOR_CP +#define TICKS_PER_uSEC 1 +#else #define TICKS_PER_uSEC 24 +#endif /* * These are useconds NOT ticks. diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/timex.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/timex.h --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/timex.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/timex.h 2006-07-16 15:01:24.000000000 +0200 @@ -21,6 +21,6 @@ */ /* - * ?? + * Timer rate */ -#define CLOCK_TICK_RATE (50000000 / 16) +#define CLOCK_TICK_RATE (1000000) diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/atomic.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/atomic.h --- linux-2.6.16.5-tcl1/include/asm-arm/atomic.h 2006-05-07 16:42:05.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/atomic.h 2006-07-16 15:01:24.000000000 +0200 @@ -129,10 +129,10 @@ static inline int atomic_add_return(int unsigned long flags; int val; - local_irq_save(flags); + local_irq_save_hw(flags); val = v->counter; v->counter = val += i; - local_irq_restore(flags); + local_irq_restore_hw(flags); return val; } @@ -142,10 +142,10 @@ static inline int atomic_sub_return(int unsigned long flags; int val; - local_irq_save(flags); + local_irq_save_hw(flags); val = v->counter; v->counter = val -= i; - local_irq_restore(flags); + local_irq_restore_hw(flags); return val; } @@ -155,11 +155,11 @@ static inline int atomic_cmpxchg(atomic_ int ret; unsigned long flags; - local_irq_save(flags); + local_irq_save_hw(flags); ret = v->counter; if (likely(ret == old)) v->counter = new; - local_irq_restore(flags); + local_irq_restore_hw(flags); return ret; } @@ -168,9 +168,9 @@ static inline void atomic_clear_mask(uns { unsigned long flags; - local_irq_save(flags); + local_irq_save_hw(flags); *addr &= ~mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); } #endif /* __LINUX_ARM_ARCH__ */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/bitops.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/bitops.h --- linux-2.6.16.5-tcl1/include/asm-arm/bitops.h 2006-05-07 16:42:05.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/bitops.h 2006-07-16 15:01:24.000000000 +0200 @@ -37,9 +37,9 @@ static inline void ____atomic_set_bit(un p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); *p |= mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); } static inline void ____atomic_clear_bit(unsigned int bit, volatile unsigned long *p) @@ -49,9 +49,9 @@ static inline void ____atomic_clear_bit( p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); *p &= ~mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); } static inline void ____atomic_change_bit(unsigned int bit, volatile unsigned long *p) @@ -61,9 +61,9 @@ static inline void ____atomic_change_bit p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); *p ^= mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); } static inline int @@ -75,10 +75,10 @@ ____atomic_test_and_set_bit(unsigned int p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); res = *p; *p = res | mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); return res & mask; } @@ -92,10 +92,10 @@ ____atomic_test_and_clear_bit(unsigned i p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); res = *p; *p = res & ~mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); return res & mask; } @@ -109,10 +109,10 @@ ____atomic_test_and_change_bit(unsigned p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); res = *p; *p = res ^ mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); return res & mask; } diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h --- linux-2.6.16.5-tcl1/include/asm-arm/ipipe.h 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,193 @@ +/* -*- linux-c -*- + * include/asm-arm/ipipe.h + * + * Copyright (C) 2002-2005 Philippe Gerum. + * Copyright (C) 2005 Stelian Pop. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + */ + +#ifndef __ARM_IPIPE_H +#define __ARM_IPIPE_H + +#include <linux/config.h> + +#ifdef CONFIG_IPIPE + +#include <linux/list.h> +#include <linux/cpumask.h> +#include <linux/threads.h> +#include <asm/ptrace.h> +#include <asm/irq.h> +#include <asm/bitops.h> +#include <asm/mach/irq.h> + +#define IPIPE_ARCH_STRING "1.3-04" +#define IPIPE_MAJOR_NUMBER 1 +#define IPIPE_MINOR_NUMBER 3 +#define IPIPE_PATCH_NUMBER 4 + +#define IPIPE_NR_XIRQS NR_IRQS +#define IPIPE_IRQ_ISHIFT 5 /* 25 for 32bits arch. */ + +#ifdef CONFIG_SMP +#error "I-pipe/arm: SMP not yet implemented" +#define ipipe_processor_id() (current_thread_info()->cpu) +#else /* !CONFIG_SMP */ +#define ipipe_processor_id() 0 +#endif /* CONFIG_SMP */ + +/* Note that we disable the interrupts around context_switch, + * or we'll get into severe problems when scheduling Xenomaï + * user space real time threads. + * This can however cause high latencies, see for example: + * http://www.ussg.iu.edu/hypermail/linux/kernel/0405.2/1388.html + * This may need further optimization... + */ +#define prepare_arch_switch(next) \ +do { \ + __ipipe_dispatch_event(IPIPE_EVENT_SCHEDULE,next); \ + local_irq_disable_hw(); \ +} while(0) + +#define task_hijacked(p) \ + ( { \ + int x = ipipe_current_domain != ipipe_root_domain; \ + __clear_bit(IPIPE_SYNC_FLAG, \ + &ipipe_root_domain->cpudata[task_cpu(p)].status); \ + local_irq_enable_hw(); \ + x; \ + } ) + +/* ARM traps */ +#define IPIPE_TRAP_ACCESS 0 /* Data or instruction access exception */ +#define IPIPE_TRAP_SECTION 1 /* Section fault */ +#define IPIPE_TRAP_DABT 2 /* Generic data abort */ +#define IPIPE_TRAP_UNKNOWN 3 /* Unknown exception */ +#define IPIPE_TRAP_BREAK 4 /* Instruction breakpoint */ +#define IPIPE_TRAP_FPU 5 /* Floating point exception */ +#define IPIPE_TRAP_VFP 6 /* VFP floating point exception */ +#define IPIPE_TRAP_UNDEFINSTR 7 /* Undefined instruction */ +#define IPIPE_NR_FAULTS 8 + +/* Pseudo-vectors used for kernel events */ +#define IPIPE_FIRST_EVENT IPIPE_NR_FAULTS +#define IPIPE_EVENT_SYSCALL (IPIPE_FIRST_EVENT) +#define IPIPE_EVENT_SCHEDULE (IPIPE_FIRST_EVENT + 1) +#define IPIPE_EVENT_SIGWAKE (IPIPE_FIRST_EVENT + 2) +#define IPIPE_EVENT_SETSCHED (IPIPE_FIRST_EVENT + 3) +#define IPIPE_EVENT_EXIT (IPIPE_FIRST_EVENT + 4) +#define IPIPE_EVENT_CLEANUP (IPIPE_FIRST_EVENT + 5) +#define IPIPE_LAST_EVENT IPIPE_EVENT_CLEANUP +#define IPIPE_NR_EVENTS (IPIPE_LAST_EVENT + 1) + +struct ipipe_domain; + +struct ipipe_sysinfo { + + int ncpus; /* Number of CPUs on board */ + u64 cpufreq; /* CPU frequency (in Hz) */ + + /* Arch-dependent block */ + + struct { + unsigned tmirq; /* Timer tick IRQ */ + u64 tmfreq; /* Timer frequency */ + } archdep; +}; + +/* arch specific stuff */ +extern int __ipipe_mach_timerint; +extern int __ipipe_mach_timerstolen; +extern unsigned int __ipipe_mach_ticks_per_jiffy; +extern void __ipipe_mach_acktimer(void); +extern unsigned long long __ipipe_mach_get_tsc(void); +extern void __ipipe_mach_set_dec(unsigned long); +extern unsigned long __ipipe_mach_get_dec(void); + +#define ipipe_read_tsc(t) do { t = __ipipe_mach_get_tsc(); } while (0) +#define __ipipe_read_timebase() __ipipe_mach_get_tsc() + +#define ipipe_cpu_freq() (HZ * __ipipe_mach_ticks_per_jiffy) +#define ipipe_tsc2ns(t) (((t) * 1000) / (ipipe_cpu_freq() / 1000000)) + +/* Private interface -- Internal use only */ + +#define __ipipe_check_platform() do { } while(0) + +#define __ipipe_enable_irq(irq) irq_desc[irq].chip->unmask(irq) + +#define __ipipe_disable_irq(irq) irq_desc[irq].chip->mask(irq) + +void __ipipe_init_platform(void); + +void __ipipe_enable_pipeline(void); + +int __ipipe_ack_irq(unsigned irq); + +int __ipipe_ack_timerirq(unsigned irq); + +void __ipipe_do_IRQ(int irq, + struct pt_regs *regs); + +void __ipipe_do_timer(int irq, + struct pt_regs *regs); + +void __ipipe_do_critical_sync(unsigned irq, + void *cookie); + +extern unsigned long __ipipe_decr_ticks; + +extern unsigned long long __ipipe_decr_next[]; + +extern struct pt_regs __ipipe_tick_regs[]; + +void __ipipe_handle_irq(int irq, + struct pt_regs *regs); + +#define __ipipe_tick_irq ipipe_timerint + +static inline unsigned long __ipipe_ffnz(unsigned long ul) +{ + return ffs(ul) - 1; +} + +#define __ipipe_run_isr(ipd, irq, cpuid) \ +do { \ + if (ipd == ipipe_root_domain) { \ + /* \ + * Linux handlers are called w/ hw interrupts on so \ + * that they could not defer interrupts for higher \ + * priority domains. \ + */ \ + local_irq_enable_hw(); \ + ((void (*)(unsigned, struct pt_regs *)) \ + ipd->irqs[irq].handler) (irq, __ipipe_tick_regs + cpuid); \ + local_irq_disable_hw(); \ + } else { \ + __clear_bit(IPIPE_SYNC_FLAG, &cpudata->status); \ + ipd->irqs[irq].handler(irq,ipd->irqs[irq].cookie); \ + __set_bit(IPIPE_SYNC_FLAG, &cpudata->status); \ + } \ +} while(0) + +#else /* !CONFIG_IPIPE */ + +#define task_hijacked(p) 0 + +#endif /* CONFIG_IPIPE */ + +#endif /* !__ARM_IPIPE_H */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/mmu_context.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/mmu_context.h --- linux-2.6.16.5-tcl1/include/asm-arm/mmu_context.h 2006-05-07 15:36:58.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/mmu_context.h 2006-07-16 15:01:24.000000000 +0200 @@ -82,14 +82,17 @@ static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next, struct task_struct *tsk) { - unsigned int cpu = smp_processor_id(); + unsigned int cpu = smp_processor_id_hw(); if (prev != next) { + unsigned long flags; + local_irq_save_hw_cond(flags); cpu_set(cpu, next->cpu_vm_mask); check_context(next); cpu_switch_mm(next->pgd, next); if (cache_is_vivt()) cpu_clear(cpu, prev->cpu_vm_mask); + local_irq_restore_hw_cond(flags); } } diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/pgalloc.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/pgalloc.h --- linux-2.6.16.5-tcl1/include/asm-arm/pgalloc.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/pgalloc.h 2006-07-16 15:01:24.000000000 +0200 @@ -29,6 +29,11 @@ extern void free_pgd_slow(pgd_t *pgd); #define check_pgt_cache() do { } while (0) +static inline void set_pgdir(unsigned long address, pgd_t entry) +{ + /* nop */ +} + /* * Allocate one PTE table. * diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/system.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h --- linux-2.6.16.5-tcl1/include/asm-arm/system.h 2006-05-07 16:42:05.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h 2006-07-16 16:22:55.000000000 +0200 @@ -186,30 +186,30 @@ static inline void sched_cacheflush(void */ #if __LINUX_ARM_ARCH__ >= 6 -#define local_irq_save(x) \ +#define local_irq_save_hw(x) \ ({ \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_irq_save\n" \ + "mrs %0, cpsr @ local_irq_save_hw\n" \ "cpsid i" \ : "=r" (x) : : "memory", "cc"); \ }) -#define local_irq_enable() __asm__("cpsie i @ __sti" : : : "memory", "cc") -#define local_irq_disable() __asm__("cpsid i @ __cli" : : : "memory", "cc") -#define local_fiq_enable() __asm__("cpsie f @ __stf" : : : "memory", "cc") -#define local_fiq_disable() __asm__("cpsid f @ __clf" : : : "memory", "cc") +#define local_irq_enable_hw() __asm__("cpsie i @ __sti" : : : "memory", "cc") +#define local_irq_disable_hw() __asm__("cpsid i @ __cli" : : : "memory", "cc") +#define local_fiq_enable_hw() __asm__("cpsie f @ __stf" : : : "memory", "cc") +#define local_fiq_disable_hw() __asm__("cpsid f @ __clf" : : : "memory", "cc") #else /* * Save the current interrupt enable state & disable IRQs */ -#define local_irq_save(x) \ +#define local_irq_save_hw(x) \ ({ \ unsigned long temp; \ (void) (&temp == &x); \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_irq_save\n" \ + "mrs %0, cpsr @ local_irq_save_hw\n" \ " orr %1, %0, #128\n" \ " msr cpsr_c, %1" \ : "=r" (x), "=r" (temp) \ @@ -220,11 +220,11 @@ static inline void sched_cacheflush(void /* * Enable IRQs */ -#define local_irq_enable() \ +#define local_irq_enable_hw() \ ({ \ unsigned long temp; \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_irq_enable\n" \ + "mrs %0, cpsr @ local_irq_enable_hw\n"\ " bic %0, %0, #128\n" \ " msr cpsr_c, %0" \ : "=r" (temp) \ @@ -235,11 +235,11 @@ static inline void sched_cacheflush(void /* * Disable IRQs */ -#define local_irq_disable() \ +#define local_irq_disable_hw() \ ({ \ unsigned long temp; \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_irq_disable\n" \ + "mrs %0, cpsr @ local_irq_disable_hw\n"\ " orr %0, %0, #128\n" \ " msr cpsr_c, %0" \ : "=r" (temp) \ @@ -250,7 +250,7 @@ static inline void sched_cacheflush(void /* * Enable FIQs */ -#define local_fiq_enable() \ +#define local_fiq_enable_hw() \ ({ \ unsigned long temp; \ __asm__ __volatile__( \ @@ -265,7 +265,7 @@ static inline void sched_cacheflush(void /* * Disable FIQs */ -#define local_fiq_disable() \ +#define local_fiq_disable_hw() \ ({ \ unsigned long temp; \ __asm__ __volatile__( \ @@ -282,29 +282,63 @@ static inline void sched_cacheflush(void /* * Save the current interrupt enable state. */ -#define local_save_flags(x) \ +#define local_save_flags_hw(x) \ ({ \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_save_flags" \ + "mrs %0, cpsr @ local_save_flags_hw" \ : "=r" (x) : : "memory", "cc"); \ }) /* * restore saved IRQ & FIQ state */ -#define local_irq_restore(x) \ +#define local_irq_restore_hw(x) \ __asm__ __volatile__( \ - "msr cpsr_c, %0 @ local_irq_restore\n" \ + "msr cpsr_c, %0 @ local_irq_restore_hw\n"\ : \ : "r" (x) \ : "memory", "cc") -#define irqs_disabled() \ -({ \ +#define irqs_disabled_hw() \ + ({ \ unsigned long flags; \ - local_save_flags(flags); \ + local_save_flags_hw(flags); \ (int)(flags & PSR_I_BIT); \ -}) + }) + + +#ifdef CONFIG_IPIPE + +void __ipipe_stall_root(void); +void __ipipe_unstall_root(void); +unsigned long __ipipe_test_root(void); +unsigned long __ipipe_test_and_stall_root(void); +void __ipipe_restore_root(unsigned long flags); + +/* PSR_I_BIT is bit no. 7 and is set if interrupts are _disabled_ */ +#define local_irq_save(flags) ((flags) = __ipipe_test_and_stall_root() << 7) +#define local_irq_enable() __ipipe_unstall_root() +#define local_irq_disable() __ipipe_stall_root() +#define local_fiq_enable() __ipipe_unstall_root() +#define local_fiq_disable() __ipipe_stall_root() +#define local_save_flags(flags) ((flags) = __ipipe_test_root() << 7) +#define local_irq_restore(flags) __ipipe_restore_root(flags & 0x00000080) + +#define irqs_disabled() __ipipe_test_root() + +#else /* !CONFIG_IPIPE */ + +#define local_irq_save(flags) local_irq_save_hw(flags) +#define local_irq_enable() local_irq_enable_hw() +#define local_irq_disable() local_irq_disable_hw() +#define local_fiq_enable() local_fiq_enable_hw() +#define local_fiq_disable() local_fiq_disable_hw() +#define local_save_flags(flags) local_save_flags_hw(flags) +#define local_irq_restore(flags) local_irq_restore_hw(flags) + +#define irqs_disabled() irqs_disabled_hw() + +#endif /* CONFIG_IPIPE */ #ifdef CONFIG_SMP diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/hardirq.h linux-2.6.16.5-tcl1-ipipe/include/linux/hardirq.h --- linux-2.6.16.5-tcl1/include/linux/hardirq.h 2006-05-07 16:42:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/hardirq.h 2006-07-16 15:01:24.000000000 +0200 @@ -87,8 +87,21 @@ extern void synchronize_irq(unsigned int # define synchronize_irq(irq) barrier() #endif +#ifdef CONFIG_IPIPE +#define nmi_enter() \ +do { \ + if (ipipe_current_domain == ipipe_root_domain) \ + irq_enter(); \ +} while(0) +#define nmi_exit() \ +do { \ + if (ipipe_current_domain == ipipe_root_domain) \ + sub_preempt_count(HARDIRQ_OFFSET); \ +} while(0) +#else /* !CONFIG_IPIPE */ #define nmi_enter() irq_enter() #define nmi_exit() sub_preempt_count(HARDIRQ_OFFSET) +#endif /* CONFIG_IPIPE */ struct task_struct; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/linux/ipipe.h --- linux-2.6.16.5-tcl1/include/linux/ipipe.h 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/ipipe.h 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,699 @@ +/* -*- linux-c -*- + * include/linux/ipipe.h + * + * Copyright (C) 2002-2005 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + */ + +#ifndef __LINUX_IPIPE_H +#define __LINUX_IPIPE_H + +#include <linux/config.h> +#include <linux/spinlock.h> +#include <linux/cache.h> +#include <asm/ipipe.h> + +#ifdef CONFIG_IPIPE + +#define IPIPE_VERSION_STRING IPIPE_ARCH_STRING +#define IPIPE_RELEASE_NUMBER ((IPIPE_MAJOR_NUMBER << 16) | \ + (IPIPE_MINOR_NUMBER << 8) | \ + (IPIPE_PATCH_NUMBER)) + +#ifndef BROKEN_BUILTIN_RETURN_ADDRESS +#define __BUILTIN_RETURN_ADDRESS0 ((unsigned long)__builtin_return_address(0)) +#define __BUILTIN_RETURN_ADDRESS1 ((unsigned long)__builtin_return_address(1)) +#endif /* !BUILTIN_RETURN_ADDRESS */ + +#define IPIPE_ROOT_PRIO 100 +#define IPIPE_ROOT_ID 0 +#define IPIPE_ROOT_NPTDKEYS 4 /* Must be <= BITS_PER_LONG */ + +#define IPIPE_RESET_TIMER 0x1 +#define IPIPE_GRAB_TIMER 0x2 + +/* Global domain flags */ +#define IPIPE_SPRINTK_FLAG 0 /* Synchronous printk() allowed */ +#define IPIPE_AHEAD_FLAG 1 /* Domain always heads the pipeline */ + +#define IPIPE_STALL_FLAG 0 /* Stalls a pipeline stage -- guaranteed at bit #0 */ +#define IPIPE_SYNC_FLAG 1 /* The interrupt syncer is running for the domain */ + +#define IPIPE_SYNC_MASK (1 << IPIPE_SYNC_FLAG) + +#define IPIPE_HANDLE_FLAG 0 +#define IPIPE_PASS_FLAG 1 +#define IPIPE_ENABLE_FLAG 2 +#define IPIPE_DYNAMIC_FLAG IPIPE_HANDLE_FLAG +#define IPIPE_STICKY_FLAG 3 +#define IPIPE_SYSTEM_FLAG 4 +#define IPIPE_LOCK_FLAG 5 +#define IPIPE_SHARED_FLAG 6 +#define IPIPE_WIRED_FLAG 7 +#define IPIPE_EXCLUSIVE_FLAG 8 + +#define IPIPE_HANDLE_MASK (1 << IPIPE_HANDLE_FLAG) +#define IPIPE_PASS_MASK (1 << IPIPE_PASS_FLAG) +#define IPIPE_ENABLE_MASK (1 << IPIPE_ENABLE_FLAG) +#define IPIPE_DYNAMIC_MASK IPIPE_HANDLE_MASK +#define IPIPE_STICKY_MASK (1 << IPIPE_STICKY_FLAG) +#define IPIPE_SYSTEM_MASK (1 << IPIPE_SYSTEM_FLAG) +#define IPIPE_LOCK_MASK (1 << IPIPE_LOCK_FLAG) +#define IPIPE_SHARED_MASK (1 << IPIPE_SHARED_FLAG) +#define IPIPE_WIRED_MASK (1 << IPIPE_WIRED_FLAG) +#define IPIPE_EXCLUSIVE_MASK (1 << IPIPE_EXCLUSIVE_FLAG) + +#define IPIPE_DEFAULT_MASK (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK) +#define IPIPE_STDROOT_MASK (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK|IPIPE_SYSTEM_MASK) + +#define IPIPE_EVENT_SELF 0x80000000 + +/* Number of virtual IRQs */ +#define IPIPE_NR_VIRQS BITS_PER_LONG +/* First virtual IRQ # */ +#define IPIPE_VIRQ_BASE (((IPIPE_NR_XIRQS + BITS_PER_LONG - 1) / BITS_PER_LONG) * BITS_PER_LONG) +/* Total number of IRQ slots */ +#define IPIPE_NR_IRQS (IPIPE_VIRQ_BASE + IPIPE_NR_VIRQS) +/* Number of indirect words needed to map the whole IRQ space. */ +#define IPIPE_IRQ_IWORDS ((IPIPE_NR_IRQS + BITS_PER_LONG - 1) / BITS_PER_LONG) +#define IPIPE_IRQ_IMASK (BITS_PER_LONG - 1) +#define IPIPE_IRQMASK_ANY (~0L) +#define IPIPE_IRQMASK_VIRT (IPIPE_IRQMASK_ANY << (IPIPE_VIRQ_BASE / BITS_PER_LONG)) + +#ifdef CONFIG_SMP + +#define IPIPE_NR_CPUS NR_CPUS +#define ipipe_declare_cpuid int cpuid +#define ipipe_load_cpuid() do { \ + (cpuid) = ipipe_processor_id(); \ + } while(0) +#define ipipe_lock_cpu(flags) do { \ + local_irq_save_hw(flags); \ + (cpuid) = ipipe_processor_id(); \ + } while(0) +#define ipipe_unlock_cpu(flags) local_irq_restore_hw(flags) +#define ipipe_get_cpu(flags) ipipe_lock_cpu(flags) +#define ipipe_put_cpu(flags) ipipe_unlock_cpu(flags) +#define ipipe_current_domain (ipipe_percpu_domain[ipipe_processor_id()]) + +#else /* !CONFIG_SMP */ + +#define IPIPE_NR_CPUS 1 +#define ipipe_declare_cpuid const int cpuid = 0 +#define ipipe_load_cpuid() do { } while(0) +#define ipipe_lock_cpu(flags) local_irq_save_hw(flags) +#define ipipe_unlock_cpu(flags) local_irq_restore_hw(flags) +#define ipipe_get_cpu(flags) do { (void)(flags); } while(0) +#define ipipe_put_cpu(flags) do { } while(0) +#define ipipe_current_domain (ipipe_percpu_domain[0]) + +#endif /* CONFIG_SMP */ + +#define ipipe_virtual_irq_p(irq) ((irq) >= IPIPE_VIRQ_BASE && \ + (irq) < IPIPE_NR_IRQS) + +typedef void (*ipipe_irq_handler_t)(unsigned irq, + void *cookie); + +#define IPIPE_SAME_HANDLER ((ipipe_irq_handler_t)(-1)) + +typedef int (*ipipe_irq_ackfn_t)(unsigned irq); + +typedef int (*ipipe_event_handler_t)(unsigned event, + struct ipipe_domain *from, + void *data); +struct ipipe_domain { + + struct list_head p_link; /* Link in pipeline */ + + struct ipcpudata { + unsigned long status; + unsigned long irq_pending_hi; + unsigned long irq_pending_lo[IPIPE_IRQ_IWORDS]; + struct ipirqcnt { + unsigned long pending_hits; + unsigned long total_hits; + } irq_counters[IPIPE_NR_IRQS]; + } ____cacheline_aligned_in_smp cpudata[IPIPE_NR_CPUS]; + + struct { + unsigned long control; + ipipe_irq_ackfn_t acknowledge; + ipipe_irq_handler_t handler; + void *cookie; + } ____cacheline_aligned irqs[IPIPE_NR_IRQS]; + + ipipe_event_handler_t evhand[IPIPE_NR_EVENTS]; /* Event handlers. */ + unsigned long long evself; /* Self-monitored event bits. */ + unsigned long flags; + unsigned domid; + const char *name; + int priority; + void *pdd; +}; + +#define IPIPE_HEAD_PRIORITY (-1) /* For domains always heading the pipeline */ + +struct ipipe_domain_attr { + + unsigned domid; /* Domain identifier -- Magic value set by caller */ + const char *name; /* Domain name -- Warning: won't be dup'ed! */ + int priority; /* Priority in interrupt pipeline */ + void (*entry) (void); /* Domain entry point */ + void *pdd; /* Per-domain (opaque) data pointer */ +}; + +/* The following macros must be used hw interrupts off. */ + +#define __ipipe_irq_cookie(ipd,irq) (ipd)->irqs[irq].cookie +#define __ipipe_irq_handler(ipd,irq) (ipd)->irqs[irq].handler + +#define __ipipe_cpudata_irq_hits(ipd,cpuid,irq) ((ipd)->cpudata[cpuid].irq_counters[irq].total_hits) + +#define __ipipe_set_irq_bit(ipd,cpuid,irq) \ +do { \ + if (!test_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) { \ + __set_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + __set_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[cpuid].irq_pending_hi); \ + } \ +} while(0) + +#define __ipipe_clear_pend(ipd,cpuid,irq) \ +do { \ + __clear_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + if ((ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT] == 0) \ + __clear_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[cpuid].irq_pending_hi); \ +} while(0) + +#define __ipipe_lock_irq(ipd,cpuid,irq) \ +do { \ + if (!test_and_set_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) \ + __ipipe_clear_pend(ipd,cpuid,irq); \ +} while(0) + +#define __ipipe_unlock_irq(ipd,irq) \ +do { \ + int __cpuid, __nr_cpus = num_online_cpus(); \ + if (test_and_clear_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) \ + for (__cpuid = 0; __cpuid < __nr_cpus; __cpuid++) \ + if ((ipd)->cpudata[__cpuid].irq_counters[irq].pending_hits > 0) { /* We need atomic ops next. */ \ + set_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[__cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + set_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[__cpuid].irq_pending_hi); \ + } \ +} while(0) + +#define __ipipe_clear_irq(ipd,irq) \ +do { \ + int __cpuid, __nr_cpus = num_online_cpus(); \ + clear_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control); \ + for (__cpuid = 0; __cpuid < __nr_cpus; __cpuid++) { \ + (ipd)->cpudata[__cpuid].irq_counters[irq].pending_hits = 0; \ + __ipipe_clear_pend(ipd,__cpuid,irq); \ + } \ +} while(0) + +#ifdef __RAW_SPIN_LOCK_UNLOCKED +#define spin_lock_hw(x) _raw_spin_lock(x) +#define spin_trylock_hw(x) _raw_spin_trylock(x) +#define spin_unlock_hw(x) _raw_spin_unlock(x) +#if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK) +#define write_lock_hw(x) _raw_write_lock(x) +#define write_trylock_hw(x) _raw_write_trylock(x) +#define write_unlock_hw(x) _raw_write_unlock(x) +#define read_lock_hw(x) _raw_read_lock(x) +#define read_trylock_hw(x) _raw_read_trylock(x) +#define read_unlock_hw(x) _raw_read_unlock(x) +#else /* UP non-debug */ +#define write_lock_hw(lock) do { (void)(lock); } while (0) +#define write_trylock_hw(lock) ({ (void)(lock); 1; }) +#define write_unlock_hw(lock) do { (void)(lock); } while (0) +#define read_lock_hw(lock) do { (void)(lock); } while (0) +#define read_trylock_hw(lock) ({ (void)(lock); 1; }) +#define read_unlock_hw(lock) do { (void)(lock); } while (0) +#endif /* CONFIG_SMP || CONFIG_DEBUG_SPINLOCK */ +#else /* !__RAW_SPIN_LOCK_UNLOCKED */ +#define spin_lock_hw(x) _spin_lock(x) +#define spin_unlock_hw(x) _spin_unlock(x) +#define spin_trylock_hw(x) _spin_trylock(x) +#define write_lock_hw(x) _write_lock(x) +#define write_unlock_hw(x) _write_unlock(x) +#define write_trylock_hw(x) _write_trylock(x) +#define read_lock_hw(x) _read_lock(x) +#define read_unlock_hw(x) _read_unlock(x) +#endif /* __RAW_SPIN_LOCK_UNLOCKED */ + +typedef spinlock_t ipipe_spinlock_t; +typedef rwlock_t ipipe_rwlock_t; +#define IPIPE_SPIN_LOCK_UNLOCKED SPIN_LOCK_UNLOCKED +#define IPIPE_RW_LOCK_UNLOCKED RW_LOCK_UNLOCKED + +#define spin_lock_irqsave_hw(x,flags) \ +do { \ + local_irq_save_hw(flags); \ + spin_lock_hw(x); \ +} while (0) + +#define spin_unlock_irqrestore_hw(x,flags) \ +do { \ + spin_unlock_hw(x); \ + local_irq_restore_hw(flags); \ +} while (0) + +#define spin_lock_irq_hw(x) \ +do { \ + local_irq_disable_hw(); \ + spin_lock_hw(x); \ +} while (0) + +#define spin_unlock_irq_hw(x) \ +do { \ + spin_unlock_hw(x); \ + local_irq_enable_hw(); \ +} while (0) + +#define read_lock_irqsave_hw(lock, flags) \ +do { \ + local_irq_save_hw(flags); \ + read_lock_hw(lock); \ +} while (0) + +#define read_unlock_irqrestore_hw(lock, flags) \ +do { \ + read_unlock_hw(lock); \ + local_irq_restore_hw(flags); \ +} while (0) + +#define write_lock_irqsave_hw(lock, flags) \ +do { \ + local_irq_save_hw(flags); \ + write_lock_hw(lock); \ +} while (0) + +#define write_unlock_irqrestore_hw(lock, flags) \ +do { \ + write_unlock_hw(lock); \ + local_irq_restore_hw(flags); \ +} while (0) + +extern struct ipipe_domain *ipipe_percpu_domain[], *ipipe_root_domain; + +extern unsigned __ipipe_printk_virq; + +extern unsigned long __ipipe_virtual_irq_map; + +extern struct list_head __ipipe_pipeline; + +extern ipipe_spinlock_t __ipipe_pipelock; + +extern int __ipipe_event_monitors[]; + +/* Private interface */ + +void ipipe_init(void); + +#ifdef CONFIG_PROC_FS +void ipipe_init_proc(void); + +#ifdef CONFIG_IPIPE_TRACE +void __ipipe_init_trace_proc(void); +#else /* !CONFIG_IPIPE_TRACE */ +#define __ipipe_init_trace_proc() do { } while(0) +#endif /* CONFIG_IPIPE_TRACE */ + +#else /* !CONFIG_PROC_FS */ +#define ipipe_init_proc() do { } while(0) +#endif /* CONFIG_PROC_FS */ + +void __ipipe_init_stage(struct ipipe_domain *ipd); + +void __ipipe_cleanup_domain(struct ipipe_domain *ipd); + +void __ipipe_add_domain_proc(struct ipipe_domain *ipd); + +void __ipipe_remove_domain_proc(struct ipipe_domain *ipd); + +void __ipipe_flush_printk(unsigned irq, void *cookie); + +void __ipipe_stall_root(void); + +void __ipipe_unstall_root(void); + +unsigned long __ipipe_test_root(void); + +unsigned long __ipipe_test_and_stall_root(void); + +void fastcall __ipipe_walk_pipeline(struct list_head *pos, int cpuid); + +void fastcall __ipipe_restore_root(unsigned long x); + +int fastcall __ipipe_schedule_irq(unsigned irq, struct list_head *head); + +int fastcall __ipipe_dispatch_event(unsigned event, void *data); + +int fastcall __ipipe_dispatch_wired(struct ipipe_domain *head, unsigned irq); + +void fastcall __ipipe_sync_stage(unsigned long syncmask); + +#ifndef __ipipe_sync_pipeline +#define __ipipe_sync_pipeline(syncmask) __ipipe_sync_stage(syncmask) +#endif + +#ifndef __ipipe_run_irqtail +#define __ipipe_run_irqtail() do { } while(0) +#endif + +#define __ipipe_pipeline_head_p(ipd) (&(ipd)->p_link == __ipipe_pipeline.next) + +/* + * Keep the following as a macro, so that client code could check for + * the support of the invariant pipeline head optimization. + */ +#define __ipipe_pipeline_head() list_entry(__ipipe_pipeline.next,struct ipipe_domain,p_link) + +#define __ipipe_event_pipelined_p(ev) \ + (__ipipe_event_monitors[ev] > 0 || (ipipe_current_domain->evself & (1LL << ev))) + +#ifdef CONFIG_SMP + +cpumask_t __ipipe_set_irq_affinity(unsigned irq, + cpumask_t cpumask); + +int fastcall __ipipe_send_ipi(unsigned ipi, + cpumask_t cpumask); + +#endif /* CONFIG_SMP */ + +/* Called with hw interrupts off. */ +static inline void __ipipe_switch_to(struct ipipe_domain *out, + struct ipipe_domain *in, int cpuid) +{ + void ipipe_suspend_domain(void); + + /* + * "in" is guaranteed to be closer than "out" from the head of the + * pipeline (and obviously different). + */ + + ipipe_percpu_domain[cpuid] = in; + + ipipe_suspend_domain(); /* Sync stage and propagate interrupts. */ + ipipe_load_cpuid(); /* Processor might have changed. */ + + if (ipipe_percpu_domain[cpuid] == in) + /* + * Otherwise, something has changed the current domain under + * our feet recycling the register set; do not override. + */ + ipipe_percpu_domain[cpuid] = out; +} + +static inline void ipipe_sigwake_notify(struct task_struct *p) +{ + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SIGWAKE)) + __ipipe_dispatch_event(IPIPE_EVENT_SIGWAKE,p); +} + +static inline void ipipe_setsched_notify(struct task_struct *p) +{ + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SETSCHED)) + __ipipe_dispatch_event(IPIPE_EVENT_SETSCHED,p); +} + +static inline void ipipe_exit_notify(struct task_struct *p) +{ + if (__ipipe_event_pipelined_p(IPIPE_EVENT_EXIT)) + __ipipe_dispatch_event(IPIPE_EVENT_EXIT,p); +} + +static inline int ipipe_trap_notify(int ex, struct pt_regs *regs) +{ + return __ipipe_event_pipelined_p(ex) ? __ipipe_dispatch_event(ex,regs) : 0; +} + +struct mm_struct; + +static inline void ipipe_cleanup_notify(struct mm_struct *mm) +{ + if (__ipipe_event_pipelined_p(IPIPE_EVENT_CLEANUP)) + __ipipe_dispatch_event(IPIPE_EVENT_CLEANUP,mm); +} + +/* Public interface */ + +int ipipe_register_domain(struct ipipe_domain *ipd, + struct ipipe_domain_attr *attr); + +int ipipe_unregister_domain(struct ipipe_domain *ipd); + +void ipipe_suspend_domain(void); + +int ipipe_virtualize_irq(struct ipipe_domain *ipd, + unsigned irq, + ipipe_irq_handler_t handler, + void *cookie, + ipipe_irq_ackfn_t acknowledge, + unsigned modemask); + +static inline int ipipe_share_irq(unsigned irq, + ipipe_irq_ackfn_t acknowledge) +{ + return ipipe_virtualize_irq(ipipe_current_domain, + irq, + IPIPE_SAME_HANDLER, + NULL, + acknowledge, + IPIPE_SHARED_MASK | IPIPE_HANDLE_MASK | + IPIPE_PASS_MASK); +} + +int ipipe_control_irq(unsigned irq, + unsigned clrmask, + unsigned setmask); + +unsigned ipipe_alloc_virq(void); + +int ipipe_free_virq(unsigned virq); + +int fastcall ipipe_trigger_irq(unsigned irq); + +static inline int ipipe_propagate_irq(unsigned irq) +{ + return __ipipe_schedule_irq(irq, ipipe_current_domain->p_link.next); +} + +static inline int ipipe_schedule_irq(unsigned irq) +{ + return __ipipe_schedule_irq(irq, &ipipe_current_domain->p_link); +} + +void fastcall ipipe_stall_pipeline_from(struct ipipe_domain *ipd); + +unsigned long fastcall ipipe_test_and_stall_pipeline_from(struct ipipe_domain *ipd); + +void fastcall ipipe_unstall_pipeline_from(struct ipipe_domain *ipd); + +unsigned long fastcall ipipe_test_and_unstall_pipeline_from(struct ipipe_domain *ipd); + +void fastcall ipipe_restore_pipeline_from(struct ipipe_domain *ipd, + unsigned long x); + +static inline unsigned long ipipe_test_pipeline_from(struct ipipe_domain *ipd) +{ + unsigned long flags, x; + ipipe_declare_cpuid; + + ipipe_get_cpu(flags); + x = test_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + ipipe_put_cpu(flags); + + return x; +} + +static inline void ipipe_restore_pipeline_nosync(struct ipipe_domain *ipd, + unsigned long x, int cpuid) +{ + /* + * If cpuid is current, then it must be held on entry + * (ipipe_get_cpu/local_irq_save_hw/local_irq_disable_hw). + */ + + if (x) + __set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + else + __clear_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); +} + +static inline void ipipe_stall_pipeline_head(void) +{ + ipipe_declare_cpuid; + unsigned long flags; + + ipipe_lock_cpu(flags); + __set_bit(IPIPE_STALL_FLAG, &__ipipe_pipeline_head()->cpudata[cpuid].status); +} + +static inline unsigned long ipipe_test_and_stall_pipeline_head(void) +{ + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + return __test_and_set_bit(IPIPE_STALL_FLAG, &__ipipe_pipeline_head()->cpudata[cpuid].status); +} + +void ipipe_unstall_pipeline_head(void); + +void fastcall __ipipe_restore_pipeline_head(struct ipipe_domain *head, + unsigned long x); + +static inline void ipipe_restore_pipeline_head(unsigned long x) +{ + struct ipipe_domain *head = __ipipe_pipeline_head(); + /* On some archs, __test_and_set_bit() might return different + * truth value than test_bit(), so we test the exclusive OR of + * both statuses, assuming that the lowest bit is always set in + * the truth value (if this is wrong, the failed optimization will + * be caught in __ipipe_restore_pipeline_head() if + * CONFIG_DEBUG_KERNEL is set). */ + if ((x ^ test_bit(IPIPE_STALL_FLAG, &head->cpudata[ipipe_processor_id()].status)) & 1) + __ipipe_restore_pipeline_head(head,x); +} + +#define ipipe_unstall_pipeline() \ + ipipe_unstall_pipeline_from(ipipe_current_domain) + +#define ipipe_test_and_unstall_pipeline() \ + ipipe_test_and_unstall_pipeline_from(ipipe_current_domain) + +#define ipipe_test_pipeline() \ + ipipe_test_pipeline_from(ipipe_current_domain) + +#define ipipe_test_and_stall_pipeline() \ + ipipe_test_and_stall_pipeline_from(ipipe_current_domain) + +#define ipipe_stall_pipeline() \ + ipipe_stall_pipeline_from(ipipe_current_domain) + +#define ipipe_restore_pipeline(x) \ + ipipe_restore_pipeline_from(ipipe_current_domain, (x)) + +void ipipe_init_attr(struct ipipe_domain_attr *attr); + +int ipipe_get_sysinfo(struct ipipe_sysinfo *sysinfo); + +int ipipe_tune_timer(unsigned long ns, + int flags); + +unsigned long ipipe_critical_enter(void (*syncfn) (void)); + +void ipipe_critical_exit(unsigned long flags); + +static inline void ipipe_set_printk_sync(struct ipipe_domain *ipd) +{ + set_bit(IPIPE_SPRINTK_FLAG, &ipd->flags); +} + +static inline void ipipe_set_printk_async(struct ipipe_domain *ipd) +{ + clear_bit(IPIPE_SPRINTK_FLAG, &ipd->flags); +} + +ipipe_event_handler_t ipipe_catch_event(struct ipipe_domain *ipd, + unsigned event, + ipipe_event_handler_t handler); + +cpumask_t ipipe_set_irq_affinity(unsigned irq, + cpumask_t cpumask); + +int fastcall ipipe_send_ipi(unsigned ipi, + cpumask_t cpumask); + +int ipipe_setscheduler_root(struct task_struct *p, + int policy, + int prio); + +int ipipe_reenter_root(struct task_struct *prev, + int policy, + int prio); + +int ipipe_alloc_ptdkey(void); + +int ipipe_free_ptdkey(int key); + +int fastcall ipipe_set_ptd(int key, + void *value); + +void fastcall *ipipe_get_ptd(int key); + +#define local_irq_enable_hw_cond() local_irq_enable_hw() +#define local_irq_disable_hw_cond() local_irq_disable_hw() +#define local_irq_save_hw_cond(flags) local_irq_save_hw(flags) +#define local_irq_restore_hw_cond(flags) local_irq_restore_hw(flags) +#define spin_lock_irqsave_hw_cond(lock,flags) spin_lock_irqsave_hw(lock,flags) +#define spin_unlock_irqrestore_hw_cond(lock,flags) spin_unlock_irqrestore_hw(lock,flags) +#define smp_processor_id_hw() ipipe_processor_id() + +#define ipipe_irq_lock(irq) \ + do { \ + ipipe_declare_cpuid; \ + ipipe_load_cpuid(); \ + __ipipe_lock_irq(ipipe_percpu_domain[cpuid], cpuid, irq);\ + } while(0) + +#define ipipe_irq_unlock(irq) \ + do { \ + ipipe_declare_cpuid; \ + ipipe_load_cpuid(); \ + __ipipe_unlock_irq(ipipe_percpu_domain[cpuid], irq); \ + } while(0) + +#define ipipe_root_domain_p (ipipe_current_domain == ipipe_root_domain) + +#else /* !CONFIG_IPIPE */ + +#define ipipe_init() do { } while(0) +#define ipipe_suspend_domain() do { } while(0) +#define ipipe_sigwake_notify(p) do { } while(0) +#define ipipe_setsched_notify(p) do { } while(0) +#define ipipe_exit_notify(p) do { } while(0) +#define ipipe_init_proc() do { } while(0) +#define ipipe_trap_notify(t,r) 0 +#define ipipe_cleanup_notify(mm) do { } while(0) + +#define spin_lock_hw(lock) spin_lock(lock) +#define spin_unlock_hw(lock) spin_unlock(lock) +#define spin_lock_irq_hw(lock) spin_lock_irq(lock) +#define spin_unlock_irq_hw(lock) spin_unlock_irq(lock) +#define spin_lock_irqsave_hw(lock,flags) spin_lock_irqsave(lock, flags) +#define spin_unlock_irqrestore_hw(lock,flags) spin_unlock_irqrestore(lock, flags) + +#define local_irq_enable_hw_cond() do { } while(0) +#define local_irq_disable_hw_cond() do { } while(0) +#define local_irq_save_hw_cond(flags) do { (void)(flags); } while(0) +#define local_irq_restore_hw_cond(flags) do { } while(0) +#define spin_lock_irqsave_hw_cond(lock,flags) do { (void)(flags); spin_lock(lock); } while(0) +#define spin_unlock_irqrestore_hw_cond(lock,flags) spin_unlock(lock) +#define smp_processor_id_hw() smp_processor_id() + +#define ipipe_irq_lock(irq) do { } while(0) +#define ipipe_irq_unlock(irq) do { } while(0) + +#define ipipe_root_domain_p 1 + +#endif /* CONFIG_IPIPE */ + +#endif /* !__LINUX_IPIPE_H */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/linkage.h linux-2.6.16.5-tcl1-ipipe/include/linux/linkage.h --- linux-2.6.16.5-tcl1/include/linux/linkage.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/linkage.h 2006-07-16 15:01:24.000000000 +0200 @@ -51,4 +51,8 @@ #define fastcall #endif +#ifndef notrace +#define notrace __attribute__((no_instrument_function)) +#endif + #endif diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/preempt.h linux-2.6.16.5-tcl1-ipipe/include/linux/preempt.h --- linux-2.6.16.5-tcl1/include/linux/preempt.h 2006-05-07 15:37:01.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/preempt.h 2006-07-16 15:01:24.000000000 +0200 @@ -27,29 +27,43 @@ asmlinkage void preempt_schedule(void); -#define preempt_disable() \ -do { \ - inc_preempt_count(); \ - barrier(); \ +#ifdef CONFIG_IPIPE +#include <asm/ipipe.h> +extern struct ipipe_domain *ipipe_percpu_domain[], *ipipe_root_domain; +#define ipipe_preempt_guard() (ipipe_percpu_domain[ipipe_processor_id()] == ipipe_root_domain) +#else /* !CONFIG_IPIPE */ +#define ipipe_preempt_guard() 1 +#endif /* CONFIG_IPIPE */ + +#define preempt_disable() \ +do { \ + if (ipipe_preempt_guard()) { \ + inc_preempt_count(); \ + barrier(); \ + } \ } while (0) -#define preempt_enable_no_resched() \ -do { \ - barrier(); \ - dec_preempt_count(); \ +#define preempt_enable_no_resched() \ +do { \ + if (ipipe_preempt_guard()) { \ + barrier(); \ + dec_preempt_count(); \ + } \ } while (0) -#define preempt_check_resched() \ -do { \ - if (unlikely(test_thread_flag(TIF_NEED_RESCHED))) \ - preempt_schedule(); \ +#define preempt_check_resched() \ +do { \ + if (ipipe_preempt_guard()) { \ + if (unlikely(test_thread_flag(TIF_NEED_RESCHED))) \ + preempt_schedule(); \ + } \ } while (0) -#define preempt_enable() \ -do { \ - preempt_enable_no_resched(); \ +#define preempt_enable() \ +do { \ + preempt_enable_no_resched(); \ barrier(); \ - preempt_check_resched(); \ + preempt_check_resched(); \ } while (0) #else diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/sched.h linux-2.6.16.5-tcl1-ipipe/include/linux/sched.h --- linux-2.6.16.5-tcl1/include/linux/sched.h 2006-05-07 16:42:13.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/sched.h 2006-07-16 15:06:26.000000000 +0200 @@ -4,6 +4,7 @@ #include <asm/param.h> /* for HZ */ #include <linux/config.h> +#include <linux/ipipe.h> #include <linux/capability.h> #include <linux/threads.h> #include <linux/kernel.h> @@ -129,6 +130,11 @@ extern unsigned long nr_iowait(void); #define EXIT_DEAD 32 /* in tsk->state again */ #define TASK_NONINTERACTIVE 64 +#ifdef CONFIG_IPIPE +#define TASK_ATOMICSWITCH 512 +#else /* !CONFIG_IPIPE */ +#define TASK_ATOMICSWITCH 0 +#endif /* CONFIG_IPIPE */ #define __set_task_state(tsk, state_value) \ do { (tsk)->state = (state_value); } while (0) @@ -871,6 +877,9 @@ struct task_struct { #endif atomic_t fs_excl; /* holding fs exclusive resources */ struct rcu_head rcu; +#ifdef CONFIG_IPIPE + void *ptd[IPIPE_ROOT_NPTDKEYS]; +#endif }; static inline pid_t process_group(struct task_struct *tsk) diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/init/Kconfig linux-2.6.16.5-tcl1-ipipe/init/Kconfig --- linux-2.6.16.5-tcl1/init/Kconfig 2006-05-07 16:42:14.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/init/Kconfig 2006-07-16 15:01:24.000000000 +0200 @@ -58,6 +58,7 @@ menu "General setup" config LOCALVERSION string "Local version - append to kernel release" + default "-ipipe" help Append an extra string to the end of your kernel version. This will show up when you type uname, for example. diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/init/main.c linux-2.6.16.5-tcl1-ipipe/init/main.c --- linux-2.6.16.5-tcl1/init/main.c 2006-07-15 20:06:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/init/main.c 2006-07-16 15:01:24.000000000 +0200 @@ -488,6 +488,11 @@ asmlinkage void __init start_kernel(void hrtimers_init(); softirq_init(); time_init(); + /* + * We need to wait for the interrupt and time subsystems to be + * initialized before enabling the pipeline. + */ + ipipe_init(); /* * HACK ALERT! This is early. We're enabling the console before @@ -613,6 +618,7 @@ static void __init do_basic_setup(void) #ifdef CONFIG_SYSCTL sysctl_init(); #endif + ipipe_init_proc(); do_initcalls(); } diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/Makefile linux-2.6.16.5-tcl1-ipipe/kernel/Makefile --- linux-2.6.16.5-tcl1/kernel/Makefile 2006-05-07 16:42:15.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/Makefile 2006-07-16 15:01:24.000000000 +0200 @@ -34,6 +34,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_IPIPE) += ipipe/ ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y) # According to Alan Modra <alan@domain.hid>, the -fno-omit-frame-pointer is diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/exit.c linux-2.6.16.5-tcl1-ipipe/kernel/exit.c --- linux-2.6.16.5-tcl1/kernel/exit.c 2006-05-07 16:42:15.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/exit.c 2006-07-16 15:01:24.000000000 +0200 @@ -852,6 +852,7 @@ fastcall NORET_TYPE void do_exit(long co exit_itimers(tsk->signal); acct_process(code); } + ipipe_exit_notify(tsk); exit_mm(tsk); exit_sem(tsk); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/fork.c linux-2.6.16.5-tcl1-ipipe/kernel/fork.c --- linux-2.6.16.5-tcl1/kernel/fork.c 2006-07-13 15:49:59.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/fork.c 2006-07-16 15:15:30.000000000 +0200 @@ -371,6 +371,7 @@ void fastcall __mmdrop(struct mm_struct void mmput(struct mm_struct *mm) { if (atomic_dec_and_test(&mm->mm_users)) { + ipipe_cleanup_notify(mm); exit_aio(mm); exit_mmap(mm); if (!list_empty(&mm->mmlist)) { @@ -1198,6 +1199,14 @@ static task_t *copy_process(unsigned lon spin_unlock(¤t->sighand->siglock); write_unlock_irq(&tasklist_lock); proc_fork_connector(p); +#ifdef CONFIG_IPIPE + { + int k; + + for (k = 0; k < IPIPE_ROOT_NPTDKEYS; k++) + p->ptd[k] = NULL; + } +#endif /* CONFIG_IPIPE */ return p; bad_fork_cleanup_namespace: diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/Kconfig linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Kconfig --- linux-2.6.16.5-tcl1/kernel/ipipe/Kconfig 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Kconfig 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,6 @@ +config IPIPE + bool "Interrupt pipeline" + default y + ---help--- + Activate this option if you want the interrupt pipeline to be + compiled in. diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/Makefile linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Makefile --- linux-2.6.16.5-tcl1/kernel/ipipe/Makefile 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Makefile 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,3 @@ + +obj-$(CONFIG_IPIPE) += core.o generic.o +obj-$(CONFIG_IPIPE_TRACE) += tracer.o diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/core.c linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/core.c --- linux-2.6.16.5-tcl1/kernel/ipipe/core.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/core.c 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,1044 @@ +/* -*- linux-c -*- + * linux/kernel/ipipe/core.c + * + * Copyright (C) 2002-2005 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-independent I-PIPE core support. + */ + +#include <linux/sched.h> +#include <linux/module.h> +#include <linux/kallsyms.h> +#include <linux/interrupt.h> +#ifdef CONFIG_PROC_FS +#include <linux/proc_fs.h> +#endif /* CONFIG_PROC_FS */ + +static struct ipipe_domain ipipe_root = + { .cpudata = {[0 ... IPIPE_NR_CPUS-1] = + { .status = (1<<IPIPE_STALL_FLAG) } } }; + +struct ipipe_domain *ipipe_root_domain = &ipipe_root; + +struct ipipe_domain *ipipe_percpu_domain[IPIPE_NR_CPUS] = + {[0 ... IPIPE_NR_CPUS - 1] = &ipipe_root }; + +ipipe_spinlock_t __ipipe_pipelock = IPIPE_SPIN_LOCK_UNLOCKED; + +LIST_HEAD(__ipipe_pipeline); + +unsigned long __ipipe_virtual_irq_map = 0; + +#ifdef CONFIG_PRINTK +unsigned __ipipe_printk_virq; +#endif /* CONFIG_PRINTK */ + +int __ipipe_event_monitors[IPIPE_NR_EVENTS]; + +/* + * ipipe_init() -- Initialization routine of the IPIPE layer. Called + * by the host kernel early during the boot procedure. + */ +void ipipe_init(void) +{ + struct ipipe_domain *ipd = &ipipe_root; + + __ipipe_check_platform(); /* Do platform dependent checks first. */ + + /* + * A lightweight registration code for the root domain. We are + * running on the boot CPU, hw interrupts are off, and + * secondary CPUs are still lost in space. + */ + + ipd->name = "Linux"; + ipd->domid = IPIPE_ROOT_ID; + ipd->priority = IPIPE_ROOT_PRIO; + + __ipipe_init_stage(ipd); + + INIT_LIST_HEAD(&ipd->p_link); + list_add_tail(&ipd->p_link, &__ipipe_pipeline); + + __ipipe_init_platform(); + +#ifdef CONFIG_PRINTK + __ipipe_printk_virq = ipipe_alloc_virq(); /* Cannot fail here. */ + ipd->irqs[__ipipe_printk_virq].handler = &__ipipe_flush_printk; + ipd->irqs[__ipipe_printk_virq].cookie = NULL; + ipd->irqs[__ipipe_printk_virq].acknowledge = NULL; + ipd->irqs[__ipipe_printk_virq].control = IPIPE_HANDLE_MASK; +#endif /* CONFIG_PRINTK */ + + __ipipe_enable_pipeline(); + + printk(KERN_INFO "I-pipe %s: pipeline enabled.\n", + IPIPE_VERSION_STRING); +} + +void __ipipe_init_stage(struct ipipe_domain *ipd) +{ + int cpuid, n; + + for (cpuid = 0; cpuid < IPIPE_NR_CPUS; cpuid++) { + ipd->cpudata[cpuid].irq_pending_hi = 0; + + for (n = 0; n < IPIPE_IRQ_IWORDS; n++) + ipd->cpudata[cpuid].irq_pending_lo[n] = 0; + + for (n = 0; n < IPIPE_NR_IRQS; n++) { + ipd->cpudata[cpuid].irq_counters[n].total_hits = 0; + ipd->cpudata[cpuid].irq_counters[n].pending_hits = 0; + } + } + + for (n = 0; n < IPIPE_NR_IRQS; n++) { + ipd->irqs[n].acknowledge = NULL; + ipd->irqs[n].handler = NULL; + ipd->irqs[n].control = IPIPE_PASS_MASK; /* Pass but don't handle */ + } + + for (n = 0; n < IPIPE_NR_EVENTS; n++) + ipd->evhand[n] = NULL; + + ipd->evself = 0; + +#ifdef CONFIG_SMP + ipd->irqs[IPIPE_CRITICAL_IPI].acknowledge = &__ipipe_ack_system_irq; + ipd->irqs[IPIPE_CRITICAL_IPI].handler = &__ipipe_do_critical_sync; + ipd->irqs[IPIPE_CRITICAL_IPI].cookie = NULL; + /* Immediately handle in the current domain but *never* pass */ + ipd->irqs[IPIPE_CRITICAL_IPI].control = + IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK|IPIPE_SYSTEM_MASK; +#endif /* CONFIG_SMP */ +} + +void __ipipe_stall_root(void) +{ + ipipe_declare_cpuid; + unsigned long flags; + + ipipe_get_cpu(flags); /* Care for migration. */ + + set_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); + +#ifdef CONFIG_SMP + if (!__ipipe_pipeline_head_p(ipipe_root_domain)) + ipipe_put_cpu(flags); +#else /* CONFIG_SMP */ + if (__ipipe_pipeline_head_p(ipipe_root_domain)) + local_irq_disable_hw(); +#endif /* CONFIG_SMP */ +} + +void __ipipe_cleanup_domain(struct ipipe_domain *ipd) +{ + ipipe_unstall_pipeline_from(ipd); + +#ifdef CONFIG_SMP + { + int cpu; + + for_each_online_cpu(cpu) { + while (ipd->cpudata[cpu].irq_pending_hi != 0) + cpu_relax(); + } + } +#endif /* CONFIG_SMP */ +} + +void __ipipe_unstall_root(void) +{ + ipipe_declare_cpuid; + + local_irq_disable_hw(); + + ipipe_load_cpuid(); + + __clear_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); + + if (ipipe_root_domain->cpudata[cpuid].irq_pending_hi != 0) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + + local_irq_enable_hw(); +} + +unsigned long __ipipe_test_root(void) +{ + unsigned long flags, x; + ipipe_declare_cpuid; + + ipipe_get_cpu(flags); /* Care for migration. */ + x = test_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); + ipipe_put_cpu(flags); + + return x; +} + +unsigned long __ipipe_test_and_stall_root(void) +{ + unsigned long flags, x; + ipipe_declare_cpuid; + + ipipe_get_cpu(flags); /* Care for migration. */ + x = test_and_set_bit(IPIPE_STALL_FLAG, + &ipipe_root_domain->cpudata[cpuid].status); + ipipe_put_cpu(flags); + + return x; +} + +void fastcall __ipipe_restore_root(unsigned long x) +{ + if (x) + __ipipe_stall_root(); + else + __ipipe_unstall_root(); +} + +void fastcall ipipe_stall_pipeline_from(struct ipipe_domain *ipd) +{ + ipipe_declare_cpuid; +#ifdef CONFIG_SMP + unsigned long flags; + + ipipe_lock_cpu(flags); /* Care for migration. */ + + __set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (!__ipipe_pipeline_head_p(ipd)) + ipipe_unlock_cpu(flags); +#else /* CONFIG_SMP */ + set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (__ipipe_pipeline_head_p(ipd)) + local_irq_disable_hw(); +#endif /* CONFIG_SMP */ +} + +unsigned long fastcall ipipe_test_and_stall_pipeline_from(struct ipipe_domain *ipd) +{ + ipipe_declare_cpuid; + unsigned long s; +#ifdef CONFIG_SMP + unsigned long flags; + + ipipe_lock_cpu(flags); /* Care for migration. */ + + s = __test_and_set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (!__ipipe_pipeline_head_p(ipd)) + ipipe_unlock_cpu(flags); +#else /* CONFIG_SMP */ + s = test_and_set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (__ipipe_pipeline_head_p(ipd)) + local_irq_disable_hw(); +#endif /* CONFIG_SMP */ + + return s; +} + +/* + * ipipe_unstall_pipeline_from() -- Unstall the pipeline and + * synchronize pending interrupts for a given domain. See + * __ipipe_walk_pipeline() for more information. + */ +void fastcall ipipe_unstall_pipeline_from(struct ipipe_domain *ipd) +{ + struct list_head *pos; + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + + __clear_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (ipd == ipipe_percpu_domain[cpuid]) + pos = &ipd->p_link; + else + pos = __ipipe_pipeline.next; + + __ipipe_walk_pipeline(pos, cpuid); + + if (__ipipe_pipeline_head_p(ipd)) + local_irq_enable_hw(); + else + ipipe_unlock_cpu(flags); +} + +unsigned long fastcall ipipe_test_and_unstall_pipeline_from(struct ipipe_domain *ipd) +{ + unsigned long flags, x; + ipipe_declare_cpuid; + + ipipe_get_cpu(flags); + x = test_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + ipipe_unstall_pipeline_from(ipd); + ipipe_put_cpu(flags); + + return x; +} + +void fastcall ipipe_restore_pipeline_from(struct ipipe_domain *ipd, + unsigned long x) +{ + if (x) + ipipe_stall_pipeline_from(ipd); + else + ipipe_unstall_pipeline_from(ipd); +} + +void ipipe_unstall_pipeline_head(void) +{ + struct ipipe_domain *head; + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + head = __ipipe_pipeline_head(); + __clear_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); + + if (unlikely(head->cpudata[cpuid].irq_pending_hi != 0)) { + if (likely(head == ipipe_percpu_domain[cpuid])) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + else + __ipipe_walk_pipeline(&head->p_link, cpuid); + } + + local_irq_enable_hw(); +} + +void fastcall __ipipe_restore_pipeline_head(struct ipipe_domain *head, unsigned long x) +{ + ipipe_declare_cpuid; + unsigned long flags; + + ipipe_lock_cpu(flags); + + if (x) { +#ifdef CONFIG_DEBUG_KERNEL + static int warned; + if (!warned && test_and_set_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status)) { + /* + * Already stalled albeit ipipe_restore_pipeline_head() + * should have detected it? Send a warning once.\n"); + */ + warned = 1; + printk(KERN_WARNING + "I-pipe: ipipe_restore_pipeline_head() optimization failed.\n"); + dump_stack(); + } +#else /* !CONFIG_DEBUG_KERNEL */ + set_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); +#endif /* CONFIG_DEBUG_KERNEL */ + } + else { + __clear_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); + if (unlikely(head->cpudata[cpuid].irq_pending_hi != 0)) { + if (likely(head == ipipe_percpu_domain[cpuid])) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + else + __ipipe_walk_pipeline(&head->p_link, cpuid); + } + local_irq_enable_hw(); + } +} + +/* __ipipe_walk_pipeline(): Plays interrupts pending in the log. Must + be called with local hw interrupts disabled. */ + +void fastcall __ipipe_walk_pipeline(struct list_head *pos, int cpuid) +{ + struct ipipe_domain *this_domain = ipipe_percpu_domain[cpuid]; + + while (pos != &__ipipe_pipeline) { + struct ipipe_domain *next_domain = + list_entry(pos, struct ipipe_domain, p_link); + + if (test_bit + (IPIPE_STALL_FLAG, &next_domain->cpudata[cpuid].status)) + break; /* Stalled stage -- do not go further. */ + + if (next_domain->cpudata[cpuid].irq_pending_hi != 0) { + + if (next_domain == this_domain) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + else { + __ipipe_switch_to(this_domain, next_domain, + cpuid); + + ipipe_load_cpuid(); /* Processor might have changed. */ + + if (this_domain->cpudata[cpuid]. + irq_pending_hi != 0 + && !test_bit(IPIPE_STALL_FLAG, + &this_domain->cpudata[cpuid].status)) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + } + + break; + } else if (next_domain == this_domain) + break; + + pos = next_domain->p_link.next; + } +} + +/* + * ipipe_suspend_domain() -- Suspend the current domain, switching to + * the next one which has pending work down the pipeline. + */ +void ipipe_suspend_domain(void) +{ + struct ipipe_domain *this_domain, *next_domain; + struct list_head *ln; + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + + this_domain = next_domain = ipipe_percpu_domain[cpuid]; + + __clear_bit(IPIPE_STALL_FLAG, &this_domain->cpudata[cpuid].status); + + if (this_domain->cpudata[cpuid].irq_pending_hi != 0) + goto sync_stage; + + for (;;) { + ln = next_domain->p_link.next; + + if (ln == &__ipipe_pipeline) + break; + + next_domain = list_entry(ln, struct ipipe_domain, p_link); + + if (test_bit(IPIPE_STALL_FLAG, + &next_domain->cpudata[cpuid].status)) + break; + + if (next_domain->cpudata[cpuid].irq_pending_hi == 0) + continue; + + ipipe_percpu_domain[cpuid] = next_domain; + +sync_stage: + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + + ipipe_load_cpuid(); /* Processor might have changed. */ + + if (ipipe_percpu_domain[cpuid] != next_domain) + /* + * Something has changed the current domain under our + * feet, recycling the register set; take note. + */ + this_domain = ipipe_percpu_domain[cpuid]; + } + + ipipe_percpu_domain[cpuid] = this_domain; + + ipipe_unlock_cpu(flags); +} + +/* ipipe_alloc_virq() -- Allocate a pipelined virtual/soft interrupt. + * Virtual interrupts are handled in exactly the same way than their + * hw-generated counterparts wrt pipelining. + */ +unsigned ipipe_alloc_virq(void) +{ + unsigned long flags, irq = 0; + int ipos; + + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); + + if (__ipipe_virtual_irq_map != ~0) { + ipos = ffz(__ipipe_virtual_irq_map); + set_bit(ipos, &__ipipe_virtual_irq_map); + irq = ipos + IPIPE_VIRQ_BASE; + } + + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); + + return irq; +} + +/* ipipe_virtualize_irq() -- Attach a handler (and optionally a hw + acknowledge routine) to an interrupt for a given domain. */ + +int ipipe_virtualize_irq(struct ipipe_domain *ipd, + unsigned irq, + ipipe_irq_handler_t handler, + void *cookie, + ipipe_irq_ackfn_t acknowledge, + unsigned modemask) +{ + unsigned long flags; + int err; + + if (irq >= IPIPE_NR_IRQS) + return -EINVAL; + + if (ipd->irqs[irq].control & IPIPE_SYSTEM_MASK) + return -EPERM; + + if (!test_bit(IPIPE_AHEAD_FLAG, &ipd->flags)) + /* Silently unwire interrupts for non-heading domains. */ + modemask &= ~IPIPE_WIRED_MASK; + + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); + + if (handler != NULL) { + + if (handler == IPIPE_SAME_HANDLER) { + handler = ipd->irqs[irq].handler; + cookie = ipd->irqs[irq].cookie; + + if (handler == NULL) { + err = -EINVAL; + goto unlock_and_exit; + } + } else if ((modemask & IPIPE_EXCLUSIVE_MASK) != 0 && + ipd->irqs[irq].handler != NULL) { + err = -EBUSY; + goto unlock_and_exit; + } + + if ((modemask & (IPIPE_SHARED_MASK | IPIPE_PASS_MASK)) == + IPIPE_SHARED_MASK) { + err = -EINVAL; + goto unlock_and_exit; + } + + /* Wired interrupts can only be delivered to domains + * always heading the pipeline. */ + + if ((modemask & IPIPE_WIRED_MASK) != 0) { + if ((modemask & (IPIPE_SHARED_MASK | IPIPE_PASS_MASK | IPIPE_STICKY_MASK)) != 0) { + err = -EINVAL; + goto unlock_and_exit; + } + modemask |= (IPIPE_HANDLE_MASK); + } + + if ((modemask & IPIPE_STICKY_MASK) != 0) + modemask |= IPIPE_HANDLE_MASK; + } else + modemask &= + ~(IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK | + IPIPE_SHARED_MASK | IPIPE_EXCLUSIVE_MASK | IPIPE_WIRED_MASK); + + if (acknowledge == NULL) { + if ((modemask & IPIPE_SHARED_MASK) == 0) { + if (!ipipe_virtual_irq_p(irq)) { + /* Acknowledge handler unspecified for a hw + interrupt -- this is ok in non-shared + management mode, but we will force the use + of the Linux-defined handler instead. */ + acknowledge = ipipe_root_domain->irqs[irq].acknowledge; + } + } + else { + /* A valid acknowledge handler to be called in shared mode + is required when declaring a shared IRQ. */ + err = -EINVAL; + goto unlock_and_exit; + } + } + + ipd->irqs[irq].handler = handler; + ipd->irqs[irq].cookie = cookie; + ipd->irqs[irq].acknowledge = acknowledge; + ipd->irqs[irq].control = modemask; + + if (irq < NR_IRQS && + handler != NULL && + !ipipe_virtual_irq_p(irq) && (modemask & IPIPE_ENABLE_MASK) != 0) { + if (ipd != ipipe_current_domain) { + /* IRQ enable/disable state is domain-sensitive, so we may + not change it for another domain. What is allowed + however is forcing some domain to handle an interrupt + source, by passing the proper 'ipd' descriptor which + thus may be different from ipipe_current_domain. */ + err = -EPERM; + goto unlock_and_exit; + } + + __ipipe_enable_irq(irq); + } + + err = 0; + + unlock_and_exit: + + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); + + return err; +} + +/* ipipe_control_irq() -- Change modes of a pipelined interrupt for + * the current domain. */ + +int ipipe_control_irq(unsigned irq, unsigned clrmask, unsigned setmask) +{ + struct ipipe_domain *ipd; + unsigned long flags; + + if (irq >= IPIPE_NR_IRQS) + return -EINVAL; + + ipd = ipipe_current_domain; + + if (ipd->irqs[irq].control & IPIPE_SYSTEM_MASK) + return -EPERM; + + if (((setmask | clrmask) & IPIPE_SHARED_MASK) != 0) + return -EINVAL; + + if (ipd->irqs[irq].handler == NULL) + setmask &= ~(IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK); + + if ((setmask & IPIPE_STICKY_MASK) != 0) + setmask |= IPIPE_HANDLE_MASK; + + if ((clrmask & (IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK)) != 0) /* If one goes, both go. */ + clrmask |= (IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK); + + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); + + ipd->irqs[irq].control &= ~clrmask; + ipd->irqs[irq].control |= setmask; + + if ((setmask & IPIPE_ENABLE_MASK) != 0) + __ipipe_enable_irq(irq); + else if ((clrmask & IPIPE_ENABLE_MASK) != 0) + __ipipe_disable_irq(irq); + + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); + + return 0; +} + +/* __ipipe_dispatch_event() -- Low-level event dispatcher. */ + +int fastcall __ipipe_dispatch_event (unsigned event, void *data) +{ + struct ipipe_domain *start_domain, *this_domain, *next_domain; + struct list_head *pos, *npos; + unsigned long flags; + ipipe_declare_cpuid; + int propagate = 1; + + ipipe_lock_cpu(flags); + + start_domain = this_domain = ipipe_percpu_domain[cpuid]; + + list_for_each_safe(pos,npos,&__ipipe_pipeline) { + + next_domain = list_entry(pos,struct ipipe_domain,p_link); + + /* + * Note: Domain migration may occur while running + * event or interrupt handlers, in which case the + * current register set is going to be recycled for a + * different domain than the initiating one. We do + * care for that, always tracking the current domain + * descriptor upon return from those handlers. + */ + if (next_domain->evhand[event] != NULL) { + ipipe_percpu_domain[cpuid] = next_domain; + ipipe_unlock_cpu(flags); + propagate = !next_domain->evhand[event](event,start_domain,data); + ipipe_lock_cpu(flags); + if (ipipe_percpu_domain[cpuid] != next_domain) + this_domain = ipipe_percpu_domain[cpuid]; + } + + if (next_domain != ipipe_root_domain && /* NEVER sync the root stage here. */ + next_domain->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG,&next_domain->cpudata[cpuid].status)) { + ipipe_percpu_domain[cpuid] = next_domain; + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + ipipe_load_cpuid(); + if (ipipe_percpu_domain[cpuid] != next_domain) + this_domain = ipipe_percpu_domain[cpuid]; + } + + ipipe_percpu_domain[cpuid] = this_domain; + + if (next_domain == this_domain || !propagate) + break; + } + + ipipe_unlock_cpu(flags); + + return !propagate; +} + +/* + * __ipipe_dispatch_wired -- Wired interrupt dispatcher. Wired + * interrupts are immediately and unconditionally delivered to the + * domain heading the pipeline upon receipt, and such domain must have + * been registered as an invariant head for the system (priority == + * IPIPE_HEAD_PRIORITY). The motivation for using wired interrupts is + * to get an extra-fast dispatching path for those IRQs, by relying on + * a straightforward logic based on assumptions that must always be + * true for invariant head domains. The following assumptions are + * made when dealing with such interrupts: + * + * 1- Wired interrupts are purely dynamic, i.e. the decision to + * propagate them down the pipeline must be done from the head domain + * ISR. + * 2- Wired interrupts cannot be shared or sticky. + * 3- The root domain cannot be an invariant pipeline head, in + * consequence of what the root domain cannot handle wired + * interrupts. + * 4- Wired interrupts must have a valid acknowledge handler for the + * head domain (if needed), and in any case, must not rely on handlers + * provided by lower priority domains during the acknowledge cycle + * (see __ipipe_handle_irq). + * + * Called with hw interrupts off. + */ + +int fastcall __ipipe_dispatch_wired(struct ipipe_domain *head, unsigned irq) +{ + struct ipcpudata *cpudata; + struct ipipe_domain *old; + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + cpudata = &head->cpudata[cpuid]; + cpudata->irq_counters[irq].total_hits++; + + if (test_bit(IPIPE_LOCK_FLAG, &head->irqs[irq].control)) { + /* If we can't process this IRQ right now, we must + * mark it as pending, so that it will get played + * during normal log sync when the corresponding + * interrupt source is eventually unlocked. */ + cpudata->irq_counters[irq].pending_hits++; + return 0; + } + + if (test_bit(IPIPE_STALL_FLAG, &cpudata->status)) { + cpudata->irq_counters[irq].pending_hits++; + __ipipe_set_irq_bit(head, cpuid, irq); + return 0; + } + + old = ipipe_percpu_domain[cpuid]; + ipipe_percpu_domain[cpuid] = head; /* Switch to the head domain. */ + + __set_bit(IPIPE_STALL_FLAG, &cpudata->status); + head->irqs[irq].handler(irq,head->irqs[irq].cookie); /* Call the ISR. */ + __ipipe_run_irqtail(); + __clear_bit(IPIPE_STALL_FLAG, &cpudata->status); + + /* We expect the caller to start a complete pipeline walk upon + * return, so that propagated interrupts will get played. */ + + if (ipipe_percpu_domain[cpuid] == head) + ipipe_percpu_domain[cpuid] = old; /* Back to the preempted domain. */ + + return 1; +} + +/* + * __ipipe_sync_stage() -- Flush the pending IRQs for the current + * domain (and processor). This routine flushes the interrupt log + * (see "Optimistic interrupt protection" from D. Stodolsky et al. for + * more on the deferred interrupt scheme). Every interrupt that + * occurred while the pipeline was stalled gets played. WARNING: + * callers on SMP boxen should always check for CPU migration on + * return of this routine. One can control the kind of interrupts + * which are going to be sync'ed using the syncmask + * parameter. IPIPE_IRQMASK_ANY plays them all, IPIPE_IRQMASK_VIRT + * plays virtual interrupts only. + * + * This routine must be called with hw interrupts off. + */ +void fastcall __ipipe_sync_stage(unsigned long syncmask) +{ + unsigned long mask, submask; + struct ipcpudata *cpudata; + struct ipipe_domain *ipd; + ipipe_declare_cpuid; + int level, rank; + unsigned irq; + + ipipe_load_cpuid(); + ipd = ipipe_percpu_domain[cpuid]; + cpudata = &ipd->cpudata[cpuid]; + + if (__test_and_set_bit(IPIPE_SYNC_FLAG, &cpudata->status)) + return; + + /* + * The policy here is to keep the dispatching code interrupt-free + * by stalling the current stage. If the upper domain handler + * (which we call) wants to re-enable interrupts while in a safe + * portion of the code (e.g. SA_INTERRUPT flag unset for Linux's + * sigaction()), it will have to unstall (then stall again before + * returning to us!) the stage when it sees fit. + */ + while ((mask = (cpudata->irq_pending_hi & syncmask)) != 0) { + level = __ipipe_ffnz(mask); + __clear_bit(level, &cpudata->irq_pending_hi); + + while ((submask = cpudata->irq_pending_lo[level]) != 0) { + rank = __ipipe_ffnz(submask); + irq = (level << IPIPE_IRQ_ISHIFT) + rank; + + if (test_bit(IPIPE_LOCK_FLAG, &ipd->irqs[irq].control)) { + __clear_bit(rank, &cpudata->irq_pending_lo[level]); + continue; + } + + if (--cpudata->irq_counters[irq].pending_hits == 0) + __clear_bit(rank, &cpudata->irq_pending_lo[level]); + + __set_bit(IPIPE_STALL_FLAG, &cpudata->status); + __ipipe_run_isr(ipd, irq, cpuid); +#ifdef CONFIG_SMP + { + int _cpuid = ipipe_processor_id(); + + if (_cpuid != cpuid) { /* Handle CPU migration. */ + /* + * We expect any domain to clear the SYNC bit each + * time it switches in a new task, so that preemptions + * and/or CPU migrations (in the SMP case) over the + * ISR do not lock out the log syncer for some + * indefinite amount of time. In the Linux case, + * schedule() handles this (see kernel/sched.c). For + * this reason, we don't bother clearing it here for + * the source CPU in the migration handling case, + * since it must have scheduled another task in by + * now. + */ + cpuid = _cpuid; + cpudata = &ipd->cpudata[cpuid]; + __set_bit(IPIPE_SYNC_FLAG, &cpudata->status); + } + } +#endif /* CONFIG_SMP */ + + __clear_bit(IPIPE_STALL_FLAG, &cpudata->status); + } + } + + __clear_bit(IPIPE_SYNC_FLAG, &cpudata->status); +} + +#ifdef CONFIG_PROC_FS + +#include <linux/proc_fs.h> + +struct proc_dir_entry *ipipe_proc_root; + +static int __ipipe_version_info_proc(char *page, + char **start, + off_t off, int count, int *eof, void *data) +{ + int len = sprintf(page, "%s\n", IPIPE_VERSION_STRING); + + len -= off; + + if (len <= off + count) + *eof = 1; + + *start = page + off; + + if(len > count) + len = count; + + if(len < 0) + len = 0; + + return len; +} + +static int __ipipe_common_info_proc(char *page, + char **start, + off_t off, int count, int *eof, void *data) +{ + struct ipipe_domain *ipd = (struct ipipe_domain *)data; + unsigned long ctlbits; + unsigned irq, _irq; + char *p = page; + int len; + + spin_lock(&__ipipe_pipelock); + + if (test_bit(IPIPE_AHEAD_FLAG,&ipd->flags)) + p += sprintf(p, "Invariant head"); + else + p += sprintf(p, "Priority=%d", ipd->priority); + + p += sprintf(p, ", Id=0x%.8x\n", ipd->domid); + + irq = 0; + + while (irq < IPIPE_NR_IRQS) { + ctlbits = + (ipd->irqs[irq]. + control & (IPIPE_HANDLE_MASK | IPIPE_PASS_MASK | + IPIPE_STICKY_MASK | IPIPE_WIRED_MASK)); + if (irq >= IPIPE_NR_XIRQS && !ipipe_virtual_irq_p(irq)) { + /* + * There might be a hole between the last external + * IRQ and the first virtual one; skip it. + */ + irq++; + continue; + } + + if (ipipe_virtual_irq_p(irq) + && !test_bit(irq - IPIPE_VIRQ_BASE, + &__ipipe_virtual_irq_map)) { + /* Non-allocated virtual IRQ; skip it. */ + irq++; + continue; + } + + /* + * Attempt to group consecutive IRQ numbers having the + * same virtualization settings in a single line. + */ + + _irq = irq; + + while (++_irq < IPIPE_NR_IRQS) { + if (ipipe_virtual_irq_p(_irq) != + ipipe_virtual_irq_p(irq) + || (ipipe_virtual_irq_p(_irq) + && !test_bit(_irq - IPIPE_VIRQ_BASE, + &__ipipe_virtual_irq_map)) + || ctlbits != (ipd->irqs[_irq]. + control & (IPIPE_HANDLE_MASK | + IPIPE_PASS_MASK | + IPIPE_STICKY_MASK))) + break; + } + + if (_irq == irq + 1) + p += sprintf(p, "irq%u: ", irq); + else + p += sprintf(p, "irq%u-%u: ", irq, _irq - 1); + + /* + * Statuses are as follows: + * o "accepted" means handled _and_ passed down the pipeline. + * o "grabbed" means handled, but the interrupt might be + * terminated _or_ passed down the pipeline depending on + * what the domain handler asks for to the I-pipe. + * o "wired" is basically the same as "grabbed", except that + * the interrupt is unconditionally delivered to an invariant + * pipeline head domain. + * o "passed" means unhandled by the domain but passed + * down the pipeline. + * o "discarded" means unhandled and _not_ passed down the + * pipeline. The interrupt merely disappears from the + * current domain down to the end of the pipeline. + */ + if (ctlbits & IPIPE_HANDLE_MASK) { + if (ctlbits & IPIPE_PASS_MASK) + p += sprintf(p, "accepted"); + else if (ctlbits & IPIPE_WIRED_MASK) + p += sprintf(p, "wired"); + else + p += sprintf(p, "grabbed"); + } else if (ctlbits & IPIPE_PASS_MASK) + p += sprintf(p, "passed"); + else + p += sprintf(p, "discarded"); + + if (ctlbits & IPIPE_STICKY_MASK) + p += sprintf(p, ", sticky"); + + if (ipipe_virtual_irq_p(irq)) + p += sprintf(p, ", virtual"); + + p += sprintf(p, "\n"); + + irq = _irq; + } + + spin_unlock(&__ipipe_pipelock); + + len = p - page; + + if (len <= off + count) + *eof = 1; + + *start = page + off; + + len -= off; + + if (len > count) + len = count; + + if (len < 0) + len = 0; + + return len; +} + +void __ipipe_add_domain_proc(struct ipipe_domain *ipd) +{ + create_proc_read_entry(ipd->name,0444,ipipe_proc_root,&__ipipe_common_info_proc,ipd); +} + +void __ipipe_remove_domain_proc(struct ipipe_domain *ipd) +{ + remove_proc_entry(ipd->name,ipipe_proc_root); +} + +void ipipe_init_proc(void) +{ + ipipe_proc_root = create_proc_entry("ipipe",S_IFDIR, 0); + create_proc_read_entry("version",0444,ipipe_proc_root,&__ipipe_version_info_proc,NULL); + __ipipe_init_trace_proc(); + __ipipe_add_domain_proc(ipipe_root_domain); +} + +#endif /* CONFIG_PROC_FS */ + +EXPORT_SYMBOL(ipipe_virtualize_irq); +EXPORT_SYMBOL(ipipe_control_irq); +EXPORT_SYMBOL(ipipe_suspend_domain); +EXPORT_SYMBOL(ipipe_alloc_virq); +EXPORT_SYMBOL(ipipe_percpu_domain); +EXPORT_SYMBOL(ipipe_root_domain); +EXPORT_SYMBOL(ipipe_stall_pipeline_from); +EXPORT_SYMBOL(ipipe_test_and_stall_pipeline_from); +EXPORT_SYMBOL(ipipe_unstall_pipeline_from); +EXPORT_SYMBOL(ipipe_restore_pipeline_from); +EXPORT_SYMBOL(ipipe_test_and_unstall_pipeline_from); +EXPORT_SYMBOL(ipipe_unstall_pipeline_head); +EXPORT_SYMBOL(__ipipe_restore_pipeline_head); +EXPORT_SYMBOL(__ipipe_unstall_root); +EXPORT_SYMBOL(__ipipe_stall_root); +EXPORT_SYMBOL(__ipipe_restore_root); +EXPORT_SYMBOL(__ipipe_test_and_stall_root); +EXPORT_SYMBOL(__ipipe_test_root); +EXPORT_SYMBOL(__ipipe_dispatch_event); +EXPORT_SYMBOL(__ipipe_dispatch_wired); +EXPORT_SYMBOL(__ipipe_sync_stage); +EXPORT_SYMBOL(__ipipe_pipeline); +EXPORT_SYMBOL(__ipipe_pipelock); +EXPORT_SYMBOL(__ipipe_virtual_irq_map); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/generic.c linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/generic.c --- linux-2.6.16.5-tcl1/kernel/ipipe/generic.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/generic.c 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,396 @@ +/* -*- linux-c -*- + * linux/kernel/ipipe/generic.c + * + * Copyright (C) 2002-2005 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-independent I-PIPE services. + */ + +#include <linux/version.h> +#include <linux/module.h> +#include <linux/init.h> +#include <linux/kernel.h> +#include <linux/sched.h> +#ifdef CONFIG_PROC_FS +#include <linux/proc_fs.h> +#endif /* CONFIG_PROC_FS */ + +MODULE_DESCRIPTION("I-pipe"); +MODULE_LICENSE("GPL"); + +static int __ipipe_ptd_key_count; + +static unsigned long __ipipe_ptd_key_map; + +/* ipipe_register_domain() -- Link a new domain to the pipeline. */ + +int ipipe_register_domain(struct ipipe_domain *ipd, + struct ipipe_domain_attr *attr) +{ + struct list_head *pos; + unsigned long flags; + + if (ipipe_current_domain != ipipe_root_domain) { + printk(KERN_WARNING + "I-pipe: Only the root domain may register a new domain.\n"); + return -EPERM; + } + + if (attr->priority == IPIPE_HEAD_PRIORITY && + test_bit(IPIPE_AHEAD_FLAG,&__ipipe_pipeline_head()->flags)) + return -EAGAIN; /* Cannot override current head. */ + + flags = ipipe_critical_enter(NULL); + + list_for_each(pos, &__ipipe_pipeline) { + struct ipipe_domain *_ipd = + list_entry(pos, struct ipipe_domain, p_link); + if (_ipd->domid == attr->domid) + break; + } + + ipipe_critical_exit(flags); + + if (pos != &__ipipe_pipeline) + /* A domain with the given id already exists -- fail. */ + return -EBUSY; + + ipd->name = attr->name; + ipd->domid = attr->domid; + ipd->pdd = attr->pdd; + ipd->flags = 0; + + if (attr->priority == IPIPE_HEAD_PRIORITY) { + ipd->priority = INT_MAX; + __set_bit(IPIPE_AHEAD_FLAG,&ipd->flags); + } + else + ipd->priority = attr->priority; + + __ipipe_init_stage(ipd); + + INIT_LIST_HEAD(&ipd->p_link); + +#ifdef CONFIG_PROC_FS + __ipipe_add_domain_proc(ipd); +#endif /* CONFIG_PROC_FS */ + + flags = ipipe_critical_enter(NULL); + + list_for_each(pos, &__ipipe_pipeline) { + struct ipipe_domain *_ipd = + list_entry(pos, struct ipipe_domain, p_link); + if (ipd->priority > _ipd->priority) + break; + } + + list_add_tail(&ipd->p_link, pos); + + ipipe_critical_exit(flags); + + printk(KERN_WARNING "I-pipe: Domain %s registered.\n", ipd->name); + + /* + * Finally, allow the new domain to perform its initialization + * chores. + */ + + if (attr->entry != NULL) { + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + + ipipe_percpu_domain[cpuid] = ipd; + attr->entry(); + ipipe_percpu_domain[cpuid] = ipipe_root_domain; + + ipipe_load_cpuid(); /* Processor might have changed. */ + + if (ipipe_root_domain->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG, + &ipipe_root_domain->cpudata[cpuid].status)) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + + ipipe_unlock_cpu(flags); + } + + return 0; +} + +/* ipipe_unregister_domain() -- Remove a domain from the pipeline. */ + +int ipipe_unregister_domain(struct ipipe_domain *ipd) +{ + unsigned long flags; + + if (ipipe_current_domain != ipipe_root_domain) { + printk(KERN_WARNING + "I-pipe: Only the root domain may unregister a domain.\n"); + return -EPERM; + } + + if (ipd == ipipe_root_domain) { + printk(KERN_WARNING + "I-pipe: Cannot unregister the root domain.\n"); + return -EPERM; + } +#ifdef CONFIG_SMP + { + int nr_cpus = num_online_cpus(), _cpuid; + unsigned irq; + + /* + * In the SMP case, wait for the logged events to drain on + * other processors before eventually removing the domain + * from the pipeline. + */ + + ipipe_unstall_pipeline_from(ipd); + + flags = ipipe_critical_enter(NULL); + + for (irq = 0; irq < IPIPE_NR_IRQS; irq++) { + clear_bit(IPIPE_HANDLE_FLAG, &ipd->irqs[irq].control); + clear_bit(IPIPE_STICKY_FLAG, &ipd->irqs[irq].control); + set_bit(IPIPE_PASS_FLAG, &ipd->irqs[irq].control); + } + + ipipe_critical_exit(flags); + + for (_cpuid = 0; _cpuid < nr_cpus; _cpuid++) + for (irq = 0; irq < IPIPE_NR_IRQS; irq++) + while (ipd->cpudata[_cpuid].irq_counters[irq].pending_hits > 0) + cpu_relax(); + } +#endif /* CONFIG_SMP */ + +#ifdef CONFIG_PROC_FS + __ipipe_remove_domain_proc(ipd); +#endif /* CONFIG_PROC_FS */ + + /* + * Simply remove the domain from the pipeline and we are almost done. + */ + + flags = ipipe_critical_enter(NULL); + list_del_init(&ipd->p_link); + ipipe_critical_exit(flags); + + __ipipe_cleanup_domain(ipd); + + printk(KERN_WARNING "I-pipe: Domain %s unregistered.\n", ipd->name); + + return 0; +} + +/* + * ipipe_propagate_irq() -- Force a given IRQ propagation on behalf of + * a running interrupt handler to the next domain down the pipeline. + * ipipe_schedule_irq() -- Does almost the same as above, but attempts + * to pend the interrupt for the current domain first. + */ +int fastcall __ipipe_schedule_irq(unsigned irq, struct list_head *head) +{ + struct list_head *ln; + unsigned long flags; + ipipe_declare_cpuid; + + if (irq >= IPIPE_NR_IRQS || + (ipipe_virtual_irq_p(irq) + && !test_bit(irq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map))) + return -EINVAL; + + ipipe_lock_cpu(flags); + + ln = head; + + while (ln != &__ipipe_pipeline) { + struct ipipe_domain *ipd = + list_entry(ln, struct ipipe_domain, p_link); + + if (test_bit(IPIPE_HANDLE_FLAG, &ipd->irqs[irq].control)) { + ipd->cpudata[cpuid].irq_counters[irq].total_hits++; + ipd->cpudata[cpuid].irq_counters[irq].pending_hits++; + __ipipe_set_irq_bit(ipd, cpuid, irq); + ipipe_unlock_cpu(flags); + return 1; + } + + ln = ipd->p_link.next; + } + + ipipe_unlock_cpu(flags); + + return 0; +} + +/* ipipe_free_virq() -- Release a virtual/soft interrupt. */ + +int ipipe_free_virq(unsigned virq) +{ + if (!ipipe_virtual_irq_p(virq)) + return -EINVAL; + + clear_bit(virq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map); + + return 0; +} + +void ipipe_init_attr(struct ipipe_domain_attr *attr) +{ + attr->name = "anon"; + attr->domid = 1; + attr->entry = NULL; + attr->priority = IPIPE_ROOT_PRIO; + attr->pdd = NULL; +} + +/* + * ipipe_catch_event() -- Interpose or remove an event handler for a + * given domain. + */ +ipipe_event_handler_t ipipe_catch_event(struct ipipe_domain *ipd, + unsigned event, + ipipe_event_handler_t handler) +{ + ipipe_event_handler_t old_handler; + int self = 0; + + if (event & IPIPE_EVENT_SELF) { + event &= ~IPIPE_EVENT_SELF; + self = 1; + } + + if (event >= IPIPE_NR_EVENTS) + return NULL; + + if (!(old_handler = xchg(&ipd->evhand[event],handler))) { + if (handler) { + if (self) + ipd->evself |= (1LL << event); + else + __ipipe_event_monitors[event]++; + } + } + else if (!handler) { + if (ipd->evself & (1LL << event)) + ipd->evself &= ~(1LL << event); + else + __ipipe_event_monitors[event]--; + } else if ((ipd->evself & (1LL << event)) && !self) { + __ipipe_event_monitors[event]++; + ipd->evself &= ~(1LL << event); + } else if (!(ipd->evself & (1LL << event)) && self) { + __ipipe_event_monitors[event]--; + ipd->evself |= (1LL << event); + } + + return old_handler; +} + +cpumask_t ipipe_set_irq_affinity (unsigned irq, cpumask_t cpumask) +{ +#ifdef CONFIG_SMP + if (irq >= IPIPE_NR_XIRQS) + /* Allow changing affinity of external IRQs only. */ + return CPU_MASK_NONE; + + if (num_online_cpus() > 1) + return __ipipe_set_irq_affinity(irq,cpumask); +#endif /* CONFIG_SMP */ + + return CPU_MASK_NONE; +} + +int fastcall ipipe_send_ipi (unsigned ipi, cpumask_t cpumask) + +{ +#ifdef CONFIG_SMP + return __ipipe_send_ipi(ipi,cpumask); +#else /* !CONFIG_SMP */ + return -EINVAL; +#endif /* CONFIG_SMP */ +} + +int ipipe_alloc_ptdkey (void) +{ + unsigned long flags; + int key = -1; + + spin_lock_irqsave_hw(&__ipipe_pipelock,flags); + + if (__ipipe_ptd_key_count < IPIPE_ROOT_NPTDKEYS) { + key = ffz(__ipipe_ptd_key_map); + set_bit(key,&__ipipe_ptd_key_map); + __ipipe_ptd_key_count++; + } + + spin_unlock_irqrestore_hw(&__ipipe_pipelock,flags); + + return key; +} + +int ipipe_free_ptdkey (int key) +{ + unsigned long flags; + + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) + return -EINVAL; + + spin_lock_irqsave_hw(&__ipipe_pipelock,flags); + + if (test_and_clear_bit(key,&__ipipe_ptd_key_map)) + __ipipe_ptd_key_count--; + + spin_unlock_irqrestore_hw(&__ipipe_pipelock,flags); + + return 0; +} + +int fastcall ipipe_set_ptd (int key, void *value) + +{ + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) + return -EINVAL; + + current->ptd[key] = value; + + return 0; +} + +void fastcall *ipipe_get_ptd (int key) + +{ + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) + return NULL; + + return current->ptd[key]; +} + +EXPORT_SYMBOL(ipipe_register_domain); +EXPORT_SYMBOL(ipipe_unregister_domain); +EXPORT_SYMBOL(ipipe_free_virq); +EXPORT_SYMBOL(ipipe_init_attr); +EXPORT_SYMBOL(ipipe_catch_event); +EXPORT_SYMBOL(ipipe_alloc_ptdkey); +EXPORT_SYMBOL(ipipe_free_ptdkey); +EXPORT_SYMBOL(ipipe_set_ptd); +EXPORT_SYMBOL(ipipe_get_ptd); +EXPORT_SYMBOL(ipipe_set_irq_affinity); +EXPORT_SYMBOL(ipipe_send_ipi); +EXPORT_SYMBOL(__ipipe_schedule_irq); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/irq/handle.c linux-2.6.16.5-tcl1-ipipe/kernel/irq/handle.c --- linux-2.6.16.5-tcl1/kernel/irq/handle.c 2006-05-07 15:37:02.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/irq/handle.c 2006-07-16 15:01:24.000000000 +0200 @@ -81,6 +81,17 @@ fastcall int handle_IRQ_event(unsigned i { int ret, retval = 0, status = 0; +#ifdef CONFIG_IPIPE + /* + * If processing a timer tick, pass the original regs as + * collected during preemption and not our phony - always + * kernel-originated - frame, so that we don't wreck the + * profiling code. + */ + if (__ipipe_tick_irq == irq) + regs = __ipipe_tick_regs + smp_processor_id(); +#endif /* CONFIG_IPIPE */ + if (!(action->flags & SA_INTERRUPT)) local_irq_enable(); @@ -117,16 +128,20 @@ fastcall unsigned int __do_IRQ(unsigned /* * No locking required for CPU-local interrupts: */ +#ifndef CONFIG_IPIPE if (desc->handler->ack) desc->handler->ack(irq); +#endif /* !CONFIG_IPIPE */ action_ret = handle_IRQ_event(irq, regs, desc->action); desc->handler->end(irq); return 1; } spin_lock(&desc->lock); +#ifndef CONFIG_IPIPE if (desc->handler->ack) desc->handler->ack(irq); +#endif /* !CONFIG_IPIPE */ /* * REPLAY is when Linux resends an IRQ that was dropped earlier * WAITING is used by probe to mark irqs that are being tested diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/printk.c linux-2.6.16.5-tcl1-ipipe/kernel/printk.c --- linux-2.6.16.5-tcl1/kernel/printk.c 2006-07-15 20:06:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/printk.c 2006-07-16 15:01:24.000000000 +0200 @@ -520,6 +520,78 @@ __attribute__((weak)) unsigned long long * printf(3) */ +#ifdef CONFIG_IPIPE + +static ipipe_spinlock_t __ipipe_printk_lock = IPIPE_SPIN_LOCK_UNLOCKED; + +static int __ipipe_printk_fill; + +static char __ipipe_printk_buf[__LOG_BUF_LEN]; + +void __ipipe_flush_printk (unsigned virq, void *cookie) +{ + char *p = __ipipe_printk_buf; + int len, lmax, out = 0; + unsigned long flags; + + goto start; + + do { + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); + start: + lmax = __ipipe_printk_fill; + while (out < lmax) { + len = strlen(p) + 1; + printk("%s",p); + p += len; + out += len; + } + spin_lock_irqsave_hw(&__ipipe_printk_lock,flags); + } + while (__ipipe_printk_fill != lmax); + + __ipipe_printk_fill = 0; + + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); +} + +asmlinkage int printk(const char *fmt, ...) +{ + int r, fbytes, oldcount; + unsigned long flags; + va_list args; + + va_start(args, fmt); + + if (ipipe_current_domain == ipipe_root_domain || + test_bit(IPIPE_SPRINTK_FLAG,&ipipe_current_domain->flags) || + oops_in_progress) { + r = vprintk(fmt, args); + goto out; + } + + spin_lock_irqsave_hw(&__ipipe_printk_lock,flags); + + oldcount = __ipipe_printk_fill; + fbytes = __LOG_BUF_LEN - oldcount; + + if (fbytes > 1) { + r = vscnprintf(__ipipe_printk_buf + __ipipe_printk_fill, + fbytes, fmt, args) + 1; /* account for the null byte */ + __ipipe_printk_fill += r; + } else + r = 0; + + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); + + if (oldcount == 0) + ipipe_trigger_irq(__ipipe_printk_virq); +out: + va_end(args); + + return r; +} +#else /* !CONFIG_IPIPE */ asmlinkage int printk(const char *fmt, ...) { va_list args; @@ -531,6 +603,7 @@ asmlinkage int printk(const char *fmt, . return r; } +#endif /* CONFIG_IPIPE */ /* cpu currently holding logbuf_lock */ static volatile unsigned int printk_cpu = UINT_MAX; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/sched.c linux-2.6.16.5-tcl1-ipipe/kernel/sched.c --- linux-2.6.16.5-tcl1/kernel/sched.c 2006-07-13 15:49:59.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/sched.c 2006-07-16 15:01:24.000000000 +0200 @@ -2872,13 +2872,18 @@ asmlinkage void __sched schedule(void) unsigned long run_time; int cpu, idx, new_prio; +#ifdef CONFIG_IPIPE + if (unlikely(ipipe_current_domain != ipipe_root_domain)) { + goto need_resched; + } +#endif /* CONFIG_IPIPE */ /* * Test if we are atomic. Since do_exit() needs to call into * schedule() atomically, we ignore that path for now. * Otherwise, whine if we are scheduling when we should not be. */ if (likely(!current->exit_state)) { - if (unlikely(in_atomic())) { + if (unlikely(!(current->state & TASK_ATOMICSWITCH) && in_atomic())) { printk(KERN_ERR "scheduling while atomic: " "%s/0x%08x/%d\n", current->comm, preempt_count(), current->pid); @@ -2887,8 +2892,19 @@ asmlinkage void __sched schedule(void) } profile_hit(SCHED_PROFILING, __builtin_return_address(0)); + if (unlikely(current->state & TASK_ATOMICSWITCH)) { + current->state &= ~TASK_ATOMICSWITCH; + goto preemption_off; + } need_resched: preempt_disable(); +preemption_off: +#ifdef CONFIG_IPIPE + if (unlikely(ipipe_current_domain != ipipe_root_domain)) { + preempt_enable(); + return; + } +#endif /* CONFIG_IPIPE */ prev = current; release_kernel_lock(prev); need_resched_nonpreemptible: @@ -3027,6 +3043,8 @@ switch_tasks: prepare_task_switch(rq, next); prev = context_switch(rq, prev, next); barrier(); + if (task_hijacked(prev)) + return; /* * this_rq must be evaluated again because prev may have moved * CPUs since it called schedule(), thus the 'rq' on its stack @@ -3059,6 +3077,11 @@ asmlinkage void __sched preempt_schedule struct task_struct *task = current; int saved_lock_depth; #endif +#ifdef CONFIG_IPIPE + /* Do not reschedule over non-Linux domains. */ + if (ipipe_current_domain != ipipe_root_domain) + return; +#endif /* CONFIG_IPIPE */ /* * If there is a non-zero preempt_count or interrupts are disabled, * we do not want to preempt the current task. Just return.. @@ -3697,6 +3720,7 @@ recheck: deactivate_task(p, rq); oldprio = p->prio; __setscheduler(p, policy, param->sched_priority); + ipipe_setsched_notify(p); if (array) { __activate_task(p, rq); /* @@ -6165,3 +6189,50 @@ void set_curr_task(int cpu, task_t *p) } #endif + +#ifdef CONFIG_IPIPE + +int ipipe_setscheduler_root (struct task_struct *p, int policy, int prio) +{ + prio_array_t *array; + unsigned long flags; + runqueue_t *rq; + int oldprio; + + rq = task_rq_lock(p, &flags); + array = p->array; + if (array) + deactivate_task(p, rq); + oldprio = p->prio; + __setscheduler(p, policy, prio); + if (array) { + __activate_task(p, rq); + if (task_running(rq, p)) { + if (p->prio > oldprio) + resched_task(rq->curr); + } else if (TASK_PREEMPTS_CURR(p, rq)) + resched_task(rq->curr); + } + task_rq_unlock(rq, &flags); + + return 0; +} + +EXPORT_SYMBOL(ipipe_setscheduler_root); + +int ipipe_reenter_root (struct task_struct *prev, int policy, int prio) +{ + finish_task_switch(this_rq(), prev); + if (reacquire_kernel_lock(current) < 0) + ; + preempt_enable_no_resched(); + + if (current->policy != policy || current->rt_priority != prio) + return ipipe_setscheduler_root(current,policy,prio); + + return 0; +} + +EXPORT_SYMBOL(ipipe_reenter_root); + +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/signal.c linux-2.6.16.5-tcl1-ipipe/kernel/signal.c --- linux-2.6.16.5-tcl1/kernel/signal.c 2006-07-13 15:49:59.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/signal.c 2006-07-16 15:01:24.000000000 +0200 @@ -604,6 +604,7 @@ void signal_wake_up(struct task_struct * unsigned int mask; set_tsk_thread_flag(t, TIF_SIGPENDING); + ipipe_sigwake_notify(t); /* TIF_SIGPENDING must be set first. */ /* * For SIGKILL, we want to wake it up in the stopped/traced case. diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/lib/smp_processor_id.c linux-2.6.16.5-tcl1-ipipe/lib/smp_processor_id.c --- linux-2.6.16.5-tcl1/lib/smp_processor_id.c 2006-05-07 15:37:02.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/lib/smp_processor_id.c 2006-07-16 15:01:24.000000000 +0200 @@ -13,6 +13,11 @@ unsigned int debug_smp_processor_id(void int this_cpu = raw_smp_processor_id(); cpumask_t this_mask; +#ifdef CONFIG_IPIPE + if (ipipe_current_domain != ipipe_root_domain) + return this_cpu; +#endif /* CONFIG_IPIPE */ + if (likely(preempt_count)) goto out; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/lib/spinlock_debug.c linux-2.6.16.5-tcl1-ipipe/lib/spinlock_debug.c --- linux-2.6.16.5-tcl1/lib/spinlock_debug.c 2006-05-07 16:42:15.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/lib/spinlock_debug.c 2006-07-16 15:01:24.000000000 +0200 @@ -10,6 +10,7 @@ #include <linux/spinlock.h> #include <linux/interrupt.h> #include <linux/delay.h> +#include <linux/module.h> static void spin_bug(spinlock_t *lock, const char *msg) { @@ -96,6 +97,8 @@ void _raw_spin_lock(spinlock_t *lock) debug_spin_lock_after(lock); } +EXPORT_SYMBOL(_raw_spin_lock); + int _raw_spin_trylock(spinlock_t *lock) { int ret = __raw_spin_trylock(&lock->raw_lock); @@ -111,12 +114,16 @@ int _raw_spin_trylock(spinlock_t *lock) return ret; } +EXPORT_SYMBOL(_raw_spin_trylock); + void _raw_spin_unlock(spinlock_t *lock) { debug_spin_unlock(lock); __raw_spin_unlock(&lock->raw_lock); } +EXPORT_SYMBOL(_raw_spin_unlock); + static void rwlock_bug(rwlock_t *lock, const char *msg) { static long print_once = 1; @@ -167,6 +174,8 @@ void _raw_read_lock(rwlock_t *lock) __read_lock_debug(lock); } +EXPORT_SYMBOL(_raw_read_lock); + int _raw_read_trylock(rwlock_t *lock) { int ret = __raw_read_trylock(&lock->raw_lock); @@ -180,12 +189,16 @@ int _raw_read_trylock(rwlock_t *lock) return ret; } +EXPORT_SYMBOL(_raw_read_trylock); + void _raw_read_unlock(rwlock_t *lock) { RWLOCK_BUG_ON(lock->magic != RWLOCK_MAGIC, lock, "bad magic"); __raw_read_unlock(&lock->raw_lock); } +EXPORT_SYMBOL(_raw_read_unlock); + static inline void debug_write_lock_before(rwlock_t *lock) { RWLOCK_BUG_ON(lock->magic != RWLOCK_MAGIC, lock, "bad magic"); @@ -241,6 +254,8 @@ void _raw_write_lock(rwlock_t *lock) debug_write_lock_after(lock); } +EXPORT_SYMBOL(_raw_write_lock); + int _raw_write_trylock(rwlock_t *lock) { int ret = __raw_write_trylock(&lock->raw_lock); @@ -256,8 +271,12 @@ int _raw_write_trylock(rwlock_t *lock) return ret; } +EXPORT_SYMBOL(_raw_write_trylock); + void _raw_write_unlock(rwlock_t *lock) { debug_write_unlock(lock); __raw_write_unlock(&lock->raw_lock); } + +EXPORT_SYMBOL(_raw_write_unlock); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/mm/vmalloc.c linux-2.6.16.5-tcl1-ipipe/mm/vmalloc.c --- linux-2.6.16.5-tcl1/mm/vmalloc.c 2006-05-07 15:37:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/mm/vmalloc.c 2006-07-16 15:01:24.000000000 +0200 @@ -19,6 +19,7 @@ #include <asm/uaccess.h> #include <asm/tlbflush.h> +#include <asm/pgalloc.h> DEFINE_RWLOCK(vmlist_lock); @@ -148,10 +149,14 @@ int map_vm_area(struct vm_struct *area, BUG_ON(addr >= end); pgd = pgd_offset_k(addr); do { + pgd_t oldpgd; + memcpy(&oldpgd,pgd,sizeof(pgd_t)); next = pgd_addr_end(addr, end); err = vmap_pud_range(pgd, addr, next, prot, pages); if (err) break; + if (pgd_val(oldpgd) != pgd_val(*pgd)) + set_pgdir(addr, *pgd); } while (pgd++, addr = next, addr != end); flush_cache_vmap((unsigned long) area->addr, end); return err; ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-17 16:17 ` Gilles Chanteperdrix @ 2006-07-17 16:29 ` Philippe Gerum 2006-07-17 16:56 ` Gilles Chanteperdrix 2006-07-17 22:33 ` Danilo Levantesi 1 sibling, 1 reply; 36+ messages in thread From: Philippe Gerum @ 2006-07-17 16:29 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai On Mon, 2006-07-17 at 18:17 +0200, Gilles Chanteperdrix wrote: > Detlef Vollmann wrote: > > Gilles Chanteperdrix wrote: > > > Danilo Levantesi wrote: > > > > So what is the status of the port? Is anything working on a pxa arch? > > > > > > More recently, Detlef Vollmann seem also to have begun a port to > > > PXA. > > Unfortunately my priorities were shifted, so I'm not able to > > work on it for the next few weeks. > > > > What I've done so far is attached. > > IT'S UNTESTED!!! > > So it's nothing you can just use, but something that you can build > > on (possibly). > > I'd be happy to hear about any experience with this. > > > > What is missing is a look at entry-macro.S. > > Stelian Pop has done something for the Integrator that I don't > > really understand and therefore I can't say whether the PXA needs > > something similar. > > > > Detlef > > Hi, > > Starting from Detlef patch, I made some tests on an SA1100 based ARM, > attached is a patch that works for me, which I also adapted to PXA > without testing it. It would be nice if someone could test it on PXA. > > In order to do these tests, I had to adapt the ARM ipipe 2.6.15 patch to > linux 2.6.16, so the 2.6.16 ipipe patch is attached too for further > testing. Note that I had to make a small modification in > include/asm-arm/system.h, because the SA1100 version of __xchg uses > local_irq_restore, which use PSR_I_BIT, which is defined elsewhere, so I > replaced PSR_I_BIT with its value. I now wonder if it is the proper fix, > or if __xchg should be fixed to use local_irq_save_hw and > local_irq_restore_hw instead, as is the case for the atomic operations > defined in atomic.h and bitops.h. Definitely, it should be fixed to use hw masking ops. -- Philippe. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-17 16:29 ` Philippe Gerum @ 2006-07-17 16:56 ` Gilles Chanteperdrix 2006-07-31 8:34 ` Bart Jonkers 0 siblings, 1 reply; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-07-17 16:56 UTC (permalink / raw) To: rpm; +Cc: xenomai [-- Attachment #1: message body and .signature --] [-- Type: text/plain, Size: 483 bytes --] Philippe Gerum wrote: > On Mon, 2006-07-17 at 18:17 +0200, Gilles Chanteperdrix wrote: > > replaced PSR_I_BIT with its value. I now wonder if it is the proper fix, > > or if __xchg should be fixed to use local_irq_save_hw and > > local_irq_restore_hw instead, as is the case for the atomic operations > > defined in atomic.h and bitops.h. > > Definitely, it should be fixed to use hw masking ops. Attached a corrected patch for 2.6.16. -- Gilles Chanteperdrix. [-- Attachment #2: adeos-ipipe-2.6.16-arm-unofficial2.patch --] [-- Type: text/plain, Size: 146286 bytes --] diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/Kconfig linux-2.6.16.5-tcl1-ipipe/arch/arm/Kconfig --- linux-2.6.16.5-tcl1/arch/arm/Kconfig 2006-07-15 20:06:02.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/Kconfig 2006-07-16 15:01:24.000000000 +0200 @@ -401,6 +401,8 @@ config LOCAL_TIMERS accounting to be spread across the timer interval, preventing a "thundering herd" at every timer tick. +source "kernel/ipipe/Kconfig" + config PREEMPT bool "Preemptible Kernel (EXPERIMENTAL)" depends on EXPERIMENTAL diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/Makefile linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/Makefile --- linux-2.6.16.5-tcl1/arch/arm/kernel/Makefile 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/Makefile 2006-07-16 15:19:02.000000000 +0200 @@ -21,6 +21,7 @@ obj-$(CONFIG_ISA_DMA) += dma-isa.o obj-$(CONFIG_PCI) += bios32.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_OABI_COMPAT) += sys_oabi-compat.o +obj-$(CONFIG_IPIPE) += ipipe-core.o ipipe-root.o obj-$(CONFIG_IWMMXT) += iwmmxt.o AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-armv.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-armv.S --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-armv.S 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-armv.S 2006-07-16 15:19:54.000000000 +0200 @@ -4,6 +4,7 @@ * Copyright (C) 1996,1997,1998 Russell King. * ARM700 fix by Matthew Godbolt (linux-user@domain.hid) * nommu support by Hyok S. Choi (hyok.choi@domain.hid) + * Copyright (C) 2005 Stelian Pop. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2 as @@ -33,7 +34,11 @@ @ routine called with r0 = irq number, r1 = struct pt_regs * @ adrne lr, 1b +#ifdef CONFIG_IPIPE + bne __ipipe_grab_irq +#else bne asm_do_IRQ +#endif #ifdef CONFIG_SMP /* @@ -199,6 +204,11 @@ __irq_svc: #endif irq_handler +#ifdef CONFIG_IPIPE + cmp r0, #0 + beq __ipipe_fast_svc_irq_exit +#endif + #ifdef CONFIG_PREEMPT ldr r0, [tsk, #TI_FLAGS] @ get flags tst r0, #_TIF_NEED_RESCHED @@ -209,6 +219,9 @@ preempt_return: teq r0, r7 strne r0, [r0, -r0] @ bug() #endif +#ifdef CONFIG_IPIPE +__ipipe_fast_svc_irq_exit: +#endif ldr r0, [sp, #S_PSR] @ irqs are already disabled msr spsr_cxsf, r0 ldmia sp, {r0 - pc}^ @ load r0 - pc, cpsr @@ -237,6 +250,12 @@ svc_preempt: __und_svc: svc_entry +#ifdef CONFIG_IPIPE + mov r1, sp @ r0 = trapno, r1 = ®s + bl __ipipe_dispatch_event @ branch to trap handler + cmp r0, #0 + bne 1f +#endif /* CONFIG_IPIPE */ @ @ call emulation code, which returns using r9 if it has emulated @ the instruction, or the more conventional lr if we are to treat @@ -406,6 +425,12 @@ __irq_usr: #endif irq_handler +#ifdef CONFIG_IPIPE + cmp r0, #0 + bne __ipipe_usr_irq_continue + slow_restore_user_regs @ Fast exit path over non-root domains +__ipipe_usr_irq_continue: +#endif #ifdef CONFIG_PREEMPT ldr r0, [tsk, #TI_PREEMPT] str r8, [tsk, #TI_PREEMPT] @@ -499,8 +524,8 @@ call_fpe: mov pc, lr @ CP#8 mov pc, lr @ CP#9 #ifdef CONFIG_VFP - b do_vfp @ CP#10 (VFP) - b do_vfp @ CP#11 (VFP) + b _do_vfp @ CP#10 (VFP) + b _do_vfp @ CP#11 (VFP) #else mov pc, lr @ CP#10 (VFP) mov pc, lr @ CP#11 (VFP) @@ -511,10 +536,34 @@ call_fpe: mov pc, lr @ CP#15 (Control) do_fpe: +#ifdef CONFIG_IPIPE + mov r4, r0 + mov r0, #5 @ == IPIPE_TRAP_FPU + mov r1, sp @ r0 = trapno, r1 = ®s + bl __ipipe_dispatch_event @ branch to trap handler + cmp r0, #0 + ldrne pc, [r9] + mov r0, r4 +#endif ldr r4, .LCfp add r10, r10, #TI_FPSTATE @ r10 = workspace ldr pc, [r4] @ Call FP module USR entry point +#ifdef CONFIG_VFP +_do_vfp: +#ifdef CONFIG_IPIPE + mov r4, r0 + mov r0, #6 @ == IPIPE_TRAP_VFP + mov r1, sp @ r0 = trapno, r1 = ®s + bl __ipipe_dispatch_event @ branch to trap handler + cmp r0, #0 + ldrne pc, [r9] + mov r0, r4 +#endif + b do_vfp +#endif + + /* * The FP module is called with these registers set: * r0 = instruction diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-common.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-common.S --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-common.S 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-common.S 2006-07-16 16:15:08.000000000 +0200 @@ -2,6 +2,7 @@ * linux/arch/arm/kernel/entry-common.S * * Copyright (C) 2000 Russell King + * Copyright (C) 2005 Stelian Pop. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2 as @@ -20,13 +21,18 @@ * possible here, and this includes saving r0 back into the SVC * stack. */ +#ifdef CONFIG_IPIPE +__ipipe_ret_fast_syscall: + ldr r0, [sp, #S_R0+S_OFF] @ returned r0 + /* fall through */ +#endif ret_fast_syscall: disable_irq @ disable interrupts ldr r1, [tsk, #TI_FLAGS] tst r1, #_TIF_WORK_MASK bne fast_work_pending - @ fast_restore_user_regs +fast_restore_user_regs: ldr r1, [sp, #S_OFF + S_PSR] @ get calling cpsr ldr lr, [sp, #S_OFF + S_PC]! @ get pc msr spsr_cxsf, r1 @ save in spsr_svc @@ -35,6 +41,13 @@ ret_fast_syscall: add sp, sp, #S_FRAME_SIZE - S_PC movs pc, lr @ return & move spsr_svc into cpsr +#ifdef CONFIG_IPIPE +__ipipe_fast_exit_syscall: + ldr r0, [sp, #S_R0+S_OFF] @ returned r0 + disable_irq @ disable interrupts + b fast_restore_user_regs +#endif /* CONFIG_IPIPE */ + /* * Ok, we need to do extra processing, enter the slow path. */ @@ -62,19 +75,15 @@ ret_slow_syscall: tst r1, #_TIF_WORK_MASK bne work_pending no_work_pending: - @ slow_restore_user_regs - ldr r1, [sp, #S_PSR] @ get calling cpsr - ldr lr, [sp, #S_PC]! @ get pc - msr spsr_cxsf, r1 @ save in spsr_svc - ldmdb sp, {r0 - lr}^ @ get calling r1 - lr - mov r0, r0 - add sp, sp, #S_FRAME_SIZE - S_PC - movs pc, lr @ return & move spsr_svc into cpsr + slow_restore_user_regs /* * This is how we return from a fork. */ ENTRY(ret_from_fork) +#ifdef CONFIG_IPIPE + enable_irq +#endif /* CONFIG_IPIPE */ bl schedule_tail get_thread_info tsk ldr r1, [tsk, #TI_FLAGS] @ check for syscall tracing @@ -197,8 +206,18 @@ ENTRY(vector_swi) bic scno, scno, #0xff000000 @ mask off SWI op-code eor scno, scno, #__NR_SYSCALL_BASE @ check OS number #endif - stmdb sp!, {r4, r5} @ push fifth and sixth args +#ifdef CONFIG_IPIPE + stmfd sp!, {r0-r3, ip} + add r1, sp, #S_OFF + add r1, r1, #20 + mov r0, scno + bl __ipipe_syscall_root + cmp r0, #0 + ldmfd sp!, {r0-r3, ip} + blt __ipipe_ret_fast_syscall + bgt __ipipe_fast_exit_syscall +#endif /* CONFIG_IPIPE */ tst ip, #_TIF_SYSCALL_TRACE @ are we tracing syscalls? bne __sys_trace @@ -245,6 +264,9 @@ __sys_trace_return: __cr_alignment: .word cr_alignment #endif +#ifdef CONFIG_IPIPE + .word __ipipe_syscall_root +#endif .ltorg /* diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-header.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-header.S --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-header.S 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-header.S 2006-07-16 15:01:24.000000000 +0200 @@ -68,6 +68,15 @@ #endif .endm + .macro slow_restore_user_regs + ldr r1, [sp, #S_PSR] @ get calling cpsr + ldr lr, [sp, #S_PC]! @ get pc + msr spsr_cxsf, r1 @ save in spsr_svc + ldmdb sp, {r0 - lr}^ @ get calling r1 - lr + mov r0, r0 + add sp, sp, #S_FRAME_SIZE - S_PC + movs pc, lr @ return & move spsr_svc into cpsr + .endm /* * These are the registers used in the syscall handler, and allow us to diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-core.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-core.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-core.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-core.c 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,239 @@ +/* -*- linux-c -*- + * linux/arch/arm/kernel/ipipe-core.c + * + * Copyright (C) 2002-2005 Philippe Gerum. + * Copyright (C) 2004 Wolfgang Grandegger (Adeos/arm port over 2.4). + * Copyright (C) 2005 Heikki Lindholm (PowerPC 970 fixes). + * Copyright (C) 2005 Stelian Pop. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-dependent I-PIPE core support for ARM. + */ + +#include <linux/config.h> +#include <linux/kernel.h> +#include <linux/smp.h> +#include <linux/sched.h> +#include <linux/slab.h> +#include <linux/bitops.h> +#include <linux/interrupt.h> +#include <linux/module.h> +#include <asm/system.h> +#include <asm/atomic.h> +#include <asm/hardirq.h> +#include <asm/io.h> + +/* Current reload value for the decrementer. */ +unsigned long __ipipe_decr_ticks; + +/* Next tick date (timebase value). */ +unsigned long long __ipipe_decr_next[IPIPE_NR_CPUS]; + +struct pt_regs __ipipe_tick_regs[IPIPE_NR_CPUS]; + +#ifdef CONFIG_SMP + +static cpumask_t __ipipe_cpu_sync_map; + +static cpumask_t __ipipe_cpu_lock_map; + +static ipipe_spinlock_t __ipipe_cpu_barrier = IPIPE_SPIN_LOCK_UNLOCKED; + +static atomic_t __ipipe_critical_count = ATOMIC_INIT(0); + +static void (*__ipipe_cpu_sync) (void); + +/* Always called with hw interrupts off. */ + +void __ipipe_do_critical_sync(unsigned irq) +{ + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + + cpu_set(cpuid, __ipipe_cpu_sync_map); + + /* + * Now we are in sync with the lock requestor running on another + * CPU. Enter a spinning wait until he releases the global + * lock. + */ + spin_lock_hw(&__ipipe_cpu_barrier); + + /* Got it. Now get out. */ + + if (__ipipe_cpu_sync) + /* Call the sync routine if any. */ + __ipipe_cpu_sync(); + + spin_unlock_hw(&__ipipe_cpu_barrier); + + cpu_clear(cpuid, __ipipe_cpu_sync_map); +} + +#endif /* CONFIG_SMP */ + +/* + * ipipe_critical_enter() -- Grab the superlock excluding all CPUs + * but the current one from a critical section. This lock is used when + * we must enforce a global critical section for a single CPU in a + * possibly SMP system whichever context the CPUs are running. + */ +unsigned long ipipe_critical_enter(void (*syncfn) (void)) +{ + unsigned long flags; + + local_irq_save_hw(flags); + +#ifdef CONFIG_SMP + if (num_online_cpus() > 1) { /* We might be running a SMP-kernel on a UP box... */ + ipipe_declare_cpuid; + cpumask_t lock_map; + + ipipe_load_cpuid(); + + if (!cpu_test_and_set(cpuid, __ipipe_cpu_lock_map)) { + while (cpu_test_and_set(BITS_PER_LONG - 1, + __ipipe_cpu_lock_map)) { + int n = 0; + do { + cpu_relax(); + } while (++n < cpuid); + } + + spin_lock_hw(&__ipipe_cpu_barrier); + + __ipipe_cpu_sync = syncfn; + + /* Send the sync IPI to all processors but the current one. */ + send_IPI_allbutself(IPIPE_CRITICAL_VECTOR); + + cpus_andnot(lock_map, cpu_online_map, + __ipipe_cpu_lock_map); + + while (!cpus_equal(__ipipe_cpu_sync_map, lock_map)) + cpu_relax(); + } + + atomic_inc(&__ipipe_critical_count); + } +#endif /* CONFIG_SMP */ + + return flags; +} + +/* ipipe_critical_exit() -- Release the superlock. */ + +void ipipe_critical_exit(unsigned long flags) +{ +#ifdef CONFIG_SMP + if (num_online_cpus() > 1) { /* We might be running a SMP-kernel on a UP box... */ + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + + if (atomic_dec_and_test(&__ipipe_critical_count)) { + spin_unlock_hw(&__ipipe_cpu_barrier); + + while (!cpus_empty(__ipipe_cpu_sync_map)) + cpu_relax(); + + cpu_clear(cpuid, __ipipe_cpu_lock_map); + cpu_clear(BITS_PER_LONG - 1, __ipipe_cpu_lock_map); + } + } +#endif /* CONFIG_SMP */ + + local_irq_restore_hw(flags); +} + +void __ipipe_init_platform(void) +{ + __ipipe_decr_ticks = __ipipe_mach_ticks_per_jiffy; +} + +int ipipe_get_sysinfo(struct ipipe_sysinfo *info) +{ + info->ncpus = num_online_cpus(); + info->cpufreq = ipipe_cpu_freq(); + info->archdep.tmirq = __ipipe_mach_timerint; + info->archdep.tmfreq = info->cpufreq; + + return 0; +} + +/* + * ipipe_trigger_irq() -- Push the interrupt at front of the pipeline + * just like if it has been actually received from a hw source. Also + * works for virtual interrupts. + */ +int ipipe_trigger_irq(unsigned irq) +{ + unsigned long flags; + + if (irq >= IPIPE_NR_IRQS || + (ipipe_virtual_irq_p(irq) + && !test_bit(irq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map))) + return -EINVAL; + + local_irq_save_hw(flags); + + __ipipe_handle_irq(irq, NULL); + + local_irq_restore_hw(flags); + + return 1; +} + +static void __ipipe_set_decr(void) +{ + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + + __ipipe_decr_next[cpuid] = __ipipe_read_timebase() + __ipipe_decr_ticks; + __ipipe_mach_set_dec(__ipipe_decr_ticks); +} + +int ipipe_tune_timer(unsigned long ns, int flags) +{ + unsigned long x, ticks; + + if (flags & IPIPE_RESET_TIMER) + ticks = __ipipe_mach_ticks_per_jiffy; + else { + ticks = (ns / 1000) * (__ipipe_mach_ticks_per_jiffy) / (1000000 / HZ); + + if (ticks > __ipipe_mach_ticks_per_jiffy) + return -EINVAL; + } + + x = ipipe_critical_enter(&__ipipe_set_decr); /* Sync with all CPUs */ + __ipipe_decr_ticks = ticks; + __ipipe_set_decr(); + ipipe_critical_exit(x); + + return 0; +} + +EXPORT_SYMBOL(__ipipe_decr_ticks); +EXPORT_SYMBOL(__ipipe_decr_next); +EXPORT_SYMBOL(ipipe_critical_enter); +EXPORT_SYMBOL(ipipe_critical_exit); +EXPORT_SYMBOL(ipipe_trigger_irq); +EXPORT_SYMBOL(ipipe_get_sysinfo); +EXPORT_SYMBOL(ipipe_tune_timer); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-root.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-root.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c 2006-07-17 14:59:06.000000000 +0200 @@ -0,0 +1,379 @@ +/* -*- linux-c -*- + * linux/arch/arm/kernel/ipipe-root.c + * + * Copyright (C) 2002-2005 Philippe Gerum (Adeos/ppc port over 2.6). + * Copyright (C) 2004 Wolfgang Grandegger (Adeos/ppc port over 2.4). + * Copyright (C) 2005 Stelian Pop. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-dependent I-pipe support for ARM. + */ + +#include <linux/config.h> +#include <linux/kernel.h> +#include <linux/sched.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/interrupt.h> +#include <linux/errno.h> +#include <asm/system.h> +#include <asm/hardirq.h> +#include <asm/atomic.h> +#include <asm/io.h> +#include <asm/irq.h> +#include <asm/unistd.h> +#include <asm/mach/irq.h> +#include <asm/mmu_context.h> + +extern struct irqdesc irq_desc[]; +extern spinlock_t irq_controller_lock; +extern asmlinkage void asm_do_IRQ(unsigned int irq, struct pt_regs *regs); + +static struct irqchip __ipipe_std_irq_dtype[NR_IRQS]; + +static void __ipipe_override_irq_unmask(unsigned irq) +{ + unsigned long flags; + + local_irq_save_hw(flags); + ipipe_irq_unlock(irq); + __ipipe_std_irq_dtype[irq].unmask(irq); + local_irq_restore_hw(flags); +} + +static void __ipipe_override_irq_mask(unsigned irq) +{ + unsigned long flags; + + local_irq_save_hw(flags); + ipipe_irq_lock(irq); + __ipipe_std_irq_dtype[irq].mask(irq); + local_irq_restore_hw(flags); +} + +static void __ipipe_override_irq_mask_ack(unsigned irq) +{ + unsigned long flags; + + local_irq_save_hw(flags); + ipipe_irq_lock(irq); + __ipipe_std_irq_dtype[irq].ack(irq); + local_irq_restore_hw(flags); +} + + +static void __ipipe_enable_sync(void) +{ + __ipipe_decr_next[ipipe_processor_id()] = + __ipipe_read_timebase() + __ipipe_mach_get_dec(); +} + +/* + * __ipipe_enable_pipeline() -- We are running on the boot CPU, hw + * interrupts are off, and secondary CPUs are still lost in space. + */ +void __ipipe_enable_pipeline(void) +{ + unsigned long flags; + unsigned irq; + + flags = ipipe_critical_enter(&__ipipe_enable_sync); + + /* First, virtualize all interrupts from the root domain. */ + + for (irq = 0; irq < NR_IRQS; irq++) + ipipe_virtualize_irq(ipipe_root_domain, + irq, + (ipipe_irq_handler_t)&asm_do_IRQ, NULL, + (irq == __ipipe_mach_timerint) ? &__ipipe_ack_timerirq : &__ipipe_ack_irq, + IPIPE_HANDLE_MASK | IPIPE_PASS_MASK); + + /* + * Interpose on the IRQ control routines so we can make them + * atomic using hw masking and prevent the interrupt log from + * being untimely flushed. + */ + + for (irq = 0; irq < NR_IRQS; irq++) + __ipipe_std_irq_dtype[irq] = *irq_desc[irq].chip; + + /* + * The original controller structs are often shared, so we first + * save them all before changing any of them. Notice that we don't + * override the ack() handler since we will enforce the necessary + * setup in __ipipe_ack_irq(). + */ + + for (irq = 0; irq < NR_IRQS; irq++) { + if (irq_desc[irq].chip->mask != NULL) + irq_desc[irq].chip->mask = __ipipe_override_irq_mask; + + if (irq_desc[irq].chip->unmask != NULL) + irq_desc[irq].chip->unmask = __ipipe_override_irq_unmask; + + if (irq_desc[irq].chip->ack != NULL) + irq_desc[irq].chip->ack = __ipipe_override_irq_mask_ack; + } + + __ipipe_decr_next[ipipe_processor_id()] = + __ipipe_read_timebase() + __ipipe_mach_get_dec(); + + ipipe_critical_exit(flags); +} + +int __ipipe_ack_irq(unsigned irq) +{ + unsigned long flags; + ipipe_declare_cpuid; + + /* + * No need to mask IRQs at hw level: we are always called from + * __ipipe_handle_irq(), so interrupts are already off. We + * stall the pipeline so that spin_lock_irq*() ops won't + * unintentionally flush it, since this could cause infinite + * recursion. + */ + + ipipe_load_cpuid(); + flags = ipipe_test_and_stall_pipeline(); + preempt_disable(); + spin_lock_hw(&irq_controller_lock); + __ipipe_std_irq_dtype[irq].ack(irq); + spin_unlock_hw(&irq_controller_lock); + preempt_enable_no_resched(); + ipipe_restore_pipeline_nosync(ipipe_percpu_domain[cpuid], flags, cpuid); + + return 1; +} + +int __ipipe_ack_timerirq(unsigned irq) +{ + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + flags = ipipe_test_and_stall_pipeline(); + preempt_disable(); + spin_lock_hw(&irq_controller_lock); + __ipipe_mach_acktimer(); + __ipipe_std_irq_dtype[irq].ack(irq); + __ipipe_std_irq_dtype[irq].unmask(irq); + spin_unlock_hw(&irq_controller_lock); + preempt_enable_no_resched(); + ipipe_restore_pipeline_nosync(ipipe_percpu_domain[cpuid], flags, cpuid); + + return 1; +} + +/* + * __ipipe_handle_irq() -- IPIPE's generic IRQ handler. An optimistic + * interrupt protection log is maintained here for each domain. Hw + * interrupts are off on entry. + */ +void __ipipe_handle_irq(int irq, struct pt_regs *regs) +{ + struct ipipe_domain *this_domain, *next_domain; + struct list_head *head, *pos; + ipipe_declare_cpuid; + int m_ack, s_ack; + + m_ack = (regs == NULL); + + if (irq >= IPIPE_NR_IRQS) { + printk(KERN_ERR "I-pipe: spurious interrupt %d\n", irq); + return; + } + + ipipe_load_cpuid(); + + this_domain = ipipe_percpu_domain[cpuid]; + + if (test_bit(IPIPE_STICKY_FLAG, &this_domain->irqs[irq].control)) + head = &this_domain->p_link; + else { + head = __ipipe_pipeline.next; + next_domain = list_entry(head, struct ipipe_domain, p_link); + if (likely(test_bit(IPIPE_WIRED_FLAG, &next_domain->irqs[irq].control))) { + if (!m_ack && next_domain->irqs[irq].acknowledge != NULL) + next_domain->irqs[irq].acknowledge(irq); + if (likely(__ipipe_dispatch_wired(next_domain, irq))) + goto finalize; + return; + } + } + + /* Ack the interrupt. */ + + s_ack = m_ack; + pos = head; + + while (pos != &__ipipe_pipeline) { + next_domain = list_entry(pos, struct ipipe_domain, p_link); + + /* + * For each domain handling the incoming IRQ, mark it as + * pending in its log. + */ + if (test_bit(IPIPE_HANDLE_FLAG, + &next_domain->irqs[irq].control)) { + /* + * Domains that handle this IRQ are polled for + * acknowledging it by decreasing priority order. The + * interrupt must be made pending _first_ in the + * domain's status flags before the PIC is unlocked. + */ + + next_domain->cpudata[cpuid].irq_counters[irq].total_hits++; + next_domain->cpudata[cpuid].irq_counters[irq].pending_hits++; + __ipipe_set_irq_bit(next_domain, cpuid, irq); + + /* + * Always get the first master acknowledge available. + * Once we've got it, allow slave acknowledge + * handlers to run (until one of them stops us). + */ + if (next_domain->irqs[irq].acknowledge != NULL) { + if (!m_ack) + m_ack = next_domain->irqs[irq].acknowledge(irq); + else if (test_bit + (IPIPE_SHARED_FLAG, + &next_domain->irqs[irq].control) && !s_ack) + s_ack = next_domain->irqs[irq].acknowledge(irq); + } + } + + /* + * If the domain does not want the IRQ to be passed down the + * interrupt pipe, exit the loop now. + */ + + if (!test_bit(IPIPE_PASS_FLAG, &next_domain->irqs[irq].control)) + break; + + pos = next_domain->p_link.next; + } + +finalize: + /* + * Now walk the pipeline, yielding control to the highest + * priority domain that has pending interrupt(s) or + * immediately to the current domain if the interrupt has been + * marked as 'sticky'. This search does not go beyond the + * current domain in the pipeline. + */ + + __ipipe_walk_pipeline(head, cpuid); +} + +asmlinkage int __ipipe_grab_irq(int irq, struct pt_regs *regs) +{ + ipipe_declare_cpuid; + + if (irq == __ipipe_mach_timerint) { + + __ipipe_tick_regs[cpuid].ARM_cpsr = regs->ARM_cpsr; + __ipipe_tick_regs[cpuid].ARM_pc = regs->ARM_pc; + + if (__ipipe_decr_ticks != __ipipe_mach_ticks_per_jiffy) { + unsigned long long next_date, now; + + next_date = __ipipe_decr_next[cpuid]; + + while ((now = __ipipe_read_timebase()) >= next_date) + next_date += __ipipe_decr_ticks; + + __ipipe_mach_set_dec(next_date - now); + + __ipipe_decr_next[cpuid] = next_date; + } + } + + __ipipe_handle_irq(irq, regs); + + ipipe_load_cpuid(); + + return (ipipe_percpu_domain[cpuid] == ipipe_root_domain && + !test_bit(IPIPE_STALL_FLAG, + &ipipe_root_domain->cpudata[cpuid].status)); +} + +asmlinkage int __ipipe_check_root(struct pt_regs *regs) +{ + ipipe_declare_cpuid; + /* + * This routine is called with hw interrupts off, so no migration + * can occur while checking the identity of the current domain. + */ + ipipe_load_cpuid(); + return (ipipe_percpu_domain[cpuid] == ipipe_root_domain && + !test_bit(IPIPE_STALL_FLAG, + &ipipe_root_domain->cpudata[cpuid].status)); +} + +asmlinkage int __ipipe_syscall_root(unsigned long scno, struct pt_regs *regs) +{ + ipipe_declare_cpuid; + unsigned long flags, origr7; + + /* We use r7 to pass the syscall number to the other domains */ + origr7 = regs->ARM_r7; + regs->ARM_r7 = __NR_SYSCALL_BASE + scno; + + /* + * This routine either returns: + * 0 -- if the syscall is to be passed to Linux; + * >0 -- if the syscall should not be passed to Linux, and no + * tail work should be performed; + * <0 -- if the syscall should not be passed to Linux but the + * tail work has to be performed (for handling signals etc). + */ + + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SYSCALL) && + __ipipe_dispatch_event(IPIPE_EVENT_SYSCALL,regs) > 0) { + /* + * We might enter here over a non-root domain and exit + * over the root one as a result of the syscall + * (i.e. by recycling the register set of the current + * context across the migration), so we need to fixup + * the interrupt flag upon return too, so that + * __ipipe_unstall_iret_root() resets the correct + * stall bit on exit. + */ + if (ipipe_current_domain == ipipe_root_domain && !in_atomic()) { + /* + * Sync pending VIRQs before _TIF_NEED_RESCHED + * is tested. + */ + ipipe_lock_cpu(flags); + if ((ipipe_root_domain->cpudata[cpuid].irq_pending_hi & IPIPE_IRQMASK_VIRT) != 0) + __ipipe_sync_pipeline(IPIPE_IRQMASK_VIRT); + ipipe_unlock_cpu(flags); + regs->ARM_r7 = origr7; + return -1; + } + regs->ARM_r7 = origr7; + return 1; + } + + regs->ARM_r7 = origr7; + return 0; +} + +EXPORT_SYMBOL_GPL(show_stack); +#ifndef MULTI_CPU +EXPORT_SYMBOL_GPL(cpu_do_switch_mm); +#endif /* MULTI_CPU */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/irq.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/irq.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/irq.c 2006-07-16 15:01:24.000000000 +0200 @@ -54,10 +54,11 @@ static int noirqdebug; static volatile unsigned long irq_err_count; -static DEFINE_SPINLOCK(irq_controller_lock); +DEFINE_SPINLOCK(irq_controller_lock); static LIST_HEAD(irq_pending); struct irqdesc irq_desc[NR_IRQS]; +EXPORT_SYMBOL(irq_desc); void (*init_arch_irq)(void) __initdata = NULL; /* @@ -412,7 +413,9 @@ do_edge_IRQ(unsigned int irq, struct irq /* * Acknowledge and clear the IRQ, but don't mask it. */ +#ifndef CONFIG_IPIPE desc->chip->ack(irq); +#endif /* CONFIG_IPIPE */ /* * Mark the IRQ currently in progress. @@ -450,8 +453,10 @@ do_edge_IRQ(unsigned int irq, struct irq * currently running. Delay it. */ desc->pending = 1; +#ifndef CONFIG_IPIPE desc->chip->mask(irq); desc->chip->ack(irq); +#endif /* CONFIG_IPIPE */ } /* @@ -468,7 +473,9 @@ do_level_IRQ(unsigned int irq, struct ir /* * Acknowledge, clear _AND_ disable the interrupt. */ +#ifndef CONFIG_IPIPE desc->chip->ack(irq); +#endif /* CONFIG_IPIPE */ if (likely(!desc->disable_depth)) { kstat_cpu(cpu).irqs[irq]++; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/process.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/process.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/process.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/process.c 2006-07-16 15:01:24.000000000 +0200 @@ -89,12 +89,12 @@ static void default_idle(void) if (hlt_counter) cpu_relax(); else { - local_irq_disable(); + local_irq_disable_hw(); if (!need_resched()) { timer_dyn_reprogram(); arch_idle(); } - local_irq_enable(); + local_irq_enable_hw(); } } @@ -120,6 +120,7 @@ void cpu_idle(void) if (!idle) idle = default_idle; + ipipe_suspend_domain(); leds_event(led_idle_start); while (!need_resched()) idle(); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ptrace.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ptrace.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/ptrace.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ptrace.c 2006-07-16 15:01:24.000000000 +0200 @@ -486,6 +486,10 @@ void ptrace_break(struct task_struct *ts static int break_trap(struct pt_regs *regs, unsigned int instr) { + + if (ipipe_trap_notify(IPIPE_TRAP_BREAK,regs)) + return 0; + ptrace_break(current, regs); return 0; } diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/traps.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/traps.c --- linux-2.6.16.5-tcl1/arch/arm/kernel/traps.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/traps.c 2006-07-16 15:01:24.000000000 +0200 @@ -310,6 +310,9 @@ asmlinkage void do_undefinstr(struct pt_ } spin_unlock_irq(&undef_lock); + if (ipipe_trap_notify(IPIPE_TRAP_UNDEFINSTR,regs)) + return; + #ifdef CONFIG_DEBUG_USER if (user_debug & UDBG_UNDEFINED) { printk(KERN_INFO "%s (%d): undefined instruction: pc=%p\n", diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-integrator/core.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/core.c --- linux-2.6.16.5-tcl1/arch/arm/mach-integrator/core.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/core.c 2006-07-16 15:01:24.000000000 +0200 @@ -2,6 +2,7 @@ * linux/arch/arm/mach-integrator/core.c * * Copyright (C) 2000-2003 Deep Blue Solutions Ltd + * Copyright (C) 2005 Stelian Pop. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2, as @@ -148,53 +149,57 @@ EXPORT_SYMBOL(cm_control); /* * How long is the timer interval? */ -#define TIMER_INTERVAL (TICKS_PER_uSEC * mSEC_10) -#if TIMER_INTERVAL >= 0x100000 -#define TICKS2USECS(x) (256 * (x) / TICKS_PER_uSEC) -#elif TIMER_INTERVAL >= 0x10000 -#define TICKS2USECS(x) (16 * (x) / TICKS_PER_uSEC) -#else #define TICKS2USECS(x) ((x) / TICKS_PER_uSEC) -#endif static unsigned long timer_reload; +static unsigned long timer_interval; +static unsigned long timer_lxlost; +static int tscok; + +#ifdef CONFIG_IPIPE +int __ipipe_mach_timerint = IRQ_TIMERINT1; +static unsigned long long __ipipe_mach_tsc; +static DEFINE_SPINLOCK(timer_lock); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); +#endif /* - * Returns number of ms since last clock interrupt. Note that interrupts - * will have been disabled by do_gettimeoffset() + * Called with IRQ disabled from do_gettimeofday(). */ -unsigned long integrator_gettimeoffset(void) +static inline unsigned long integrator_getticksoffset(void) { - unsigned long ticks1, ticks2, status; + unsigned long ticks; - /* - * Get the current number of ticks. Note that there is a race - * condition between us reading the timer and checking for - * an interrupt. We get around this by ensuring that the - * counter has not reloaded between our two reads. - */ - ticks2 = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; - do { - ticks1 = ticks2; - status = __raw_readl(VA_IC_BASE + IRQ_RAW_STATUS); - ticks2 = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; - } while (ticks2 > ticks1); + if (!tscok) + return 0; - /* - * Number of ticks since last interrupt. - */ - ticks1 = timer_reload - ticks2; + ticks = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; - /* - * Interrupt pending? If so, we've reloaded once already. - */ - if (status & (1 << IRQ_TIMERINT1)) - ticks1 += timer_reload; + if (ticks > timer_reload) + ticks = 0xffff + timer_reload - ticks; + else + ticks = timer_reload - ticks; + if (timer_interval < 0x10000) + return ticks; + else if (timer_interval < 0x100000) + return ticks * 16; + else + return ticks * 256; +} + +/* + * Returns number of ms since last clock interrupt. Note that interrupts + * will have been disabled by do_gettimeoffset() + */ +unsigned long integrator_gettimeoffset(void) +{ /* * Convert the ticks to usecs */ - return TICKS2USECS(ticks1); + return TICKS2USECS(timer_lxlost + integrator_getticksoffset()); } /* @@ -205,10 +210,22 @@ integrator_timer_interrupt(int irq, void { write_seqlock(&xtime_lock); + timer_lxlost = 0; + +#ifdef CONFIG_IPIPE /* - * clear the interrupt + * If Linux is the only domain, ack the timer and reprogram it */ - writel(1, TIMER1_VA_BASE + TIMER_INTCLR); + if (!__ipipe_mach_timerstolen) { + __ipipe_mach_tsc += integrator_getticksoffset(); +#else + writel(1, TIMER1_VA_BASE + TIMER_INTCLR); +#endif + + writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); +#ifdef CONFIG_IPIPE + } +#endif /* * the clock tick routines are only processed on the @@ -239,24 +256,30 @@ static struct irqaction integrator_timer .handler = integrator_timer_interrupt, }; -/* - * Set up timer interrupt, and return the current time in seconds. - */ -void __init integrator_time_init(unsigned long reload, unsigned int ctrl) +static inline void set_dec(unsigned long reload) { - unsigned int timer_ctrl = TIMER_CTRL_ENABLE | TIMER_CTRL_PERIODIC; + unsigned int ctrl = TIMER_CTRL_ENABLE | TIMER_CTRL_IE; timer_reload = reload; - timer_ctrl |= ctrl; + timer_interval = reload; - if (timer_reload > 0x100000) { + if (timer_reload >= 0x100000) { timer_reload >>= 8; - timer_ctrl |= TIMER_CTRL_DIV256; - } else if (timer_reload > 0x010000) { + ctrl |= TIMER_CTRL_DIV256; + } else if (timer_reload >= 0x010000) { timer_reload >>= 4; - timer_ctrl |= TIMER_CTRL_DIV16; + ctrl |= TIMER_CTRL_DIV16; } + writel(ctrl, TIMER1_VA_BASE + TIMER_CTRL); + writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); +} + +/* + * Set up timer interrupt, and return the current time in seconds. + */ +void __init integrator_time_init(unsigned long reload, unsigned int ctrl) +{ /* * Initialise to a known state (all timers off) */ @@ -264,12 +287,51 @@ void __init integrator_time_init(unsigne writel(0, TIMER1_VA_BASE + TIMER_CTRL); writel(0, TIMER2_VA_BASE + TIMER_CTRL); - writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); - writel(timer_reload, TIMER1_VA_BASE + TIMER_VALUE); - writel(timer_ctrl, TIMER1_VA_BASE + TIMER_CTRL); + set_dec(reload); /* * Make irqs happen for the system timer */ setup_irq(IRQ_TIMERINT1, &integrator_timer_irq); + + tscok = 1; } + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + writel(1, TIMER1_VA_BASE + TIMER_INTCLR); +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + unsigned long long result; + unsigned long flags; + + spin_lock_irqsave_hw(&timer_lock, flags); + result = __ipipe_mach_tsc + integrator_getticksoffset(); + spin_unlock_irqrestore_hw(&timer_lock, flags); + return result; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +void __ipipe_mach_set_dec(unsigned long reload) +{ + unsigned long ticks; + unsigned long flags; + + spin_lock_irqsave_hw(&timer_lock, flags); + ticks = integrator_getticksoffset(); + __ipipe_mach_tsc += ticks; + timer_lxlost += ticks; + + set_dec(reload); + spin_unlock_irqrestore_hw(&timer_lock, flags); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return readl(TIMER1_VA_BASE + TIMER_VALUE); +} +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-integrator/integrator_cp.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c --- linux-2.6.16.5-tcl1/arch/arm/mach-integrator/integrator_cp.c 2006-05-07 16:41:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c 2006-07-16 15:01:24.000000000 +0200 @@ -2,6 +2,7 @@ * linux/arch/arm/mach-integrator/integrator_cp.c * * Copyright (C) 2003 Deep Blue Solutions Ltd + * Copyright (C) 2005 Stelian Pop. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -568,9 +569,14 @@ static void __init intcp_init(void) #define TIMER_CTRL_IE (1 << 5) /* Interrupt Enable */ +#ifdef CONFIG_IPIPE +unsigned int __ipipe_mach_ticks_per_jiffy = 1000000 * TICKS_PER_uSEC / HZ; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); +#endif + static void __init intcp_timer_init(void) { - integrator_time_init(1000000 / HZ, TIMER_CTRL_IE); + integrator_time_init(1000000 * TICKS_PER_uSEC / HZ, TIMER_CTRL_IE); } static struct sys_timer cp_timer = { diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-pxa/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mm/fault.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mm/fault.c --- linux-2.6.16.5-tcl1/arch/arm/mm/fault.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mm/fault.c 2006-07-16 15:01:24.000000000 +0200 @@ -223,6 +223,9 @@ do_page_fault(unsigned long addr, unsign struct mm_struct *mm; int fault, sig, code; + if (ipipe_trap_notify(IPIPE_TRAP_ACCESS,regs)) + return 0; + tsk = current; mm = tsk->mm; @@ -354,6 +357,9 @@ do_translation_fault(unsigned long addr, bad_area: tsk = current; + if (ipipe_trap_notify(IPIPE_TRAP_ACCESS,regs)) + return 0; + do_bad_area(tsk, tsk->active_mm, addr, fsr, regs); return 0; } @@ -366,6 +372,10 @@ static int do_sect_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) { struct task_struct *tsk = current; + + if (ipipe_trap_notify(IPIPE_TRAP_SECTION,regs)) + return 0; + do_bad_area(tsk, tsk->active_mm, addr, fsr, regs); return 0; } @@ -376,6 +386,9 @@ do_sect_fault(unsigned long addr, unsign static int do_bad(unsigned long addr, unsigned int fsr, struct pt_regs *regs) { + if (ipipe_trap_notify(IPIPE_TRAP_DABT,regs)) + return 0; + return 1; } @@ -451,6 +464,9 @@ do_DataAbort(unsigned long addr, unsigne if (!inf->fn(addr, fsr, regs)) return; + if (ipipe_trap_notify(IPIPE_TRAP_UNKNOWN,regs)) + return; + printk(KERN_ALERT "Unhandled fault: %s (0x%03x) at 0x%08lx\n", inf->name, fsr, addr); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/drivers/pci/msi.c linux-2.6.16.5-tcl1-ipipe/drivers/pci/msi.c --- linux-2.6.16.5-tcl1/drivers/pci/msi.c 2006-05-07 16:41:40.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/drivers/pci/msi.c 2006-07-16 15:01:24.000000000 +0200 @@ -151,6 +151,21 @@ static void unmask_MSI_irq(unsigned int msi_set_mask_bit(vector, 0); } +#ifdef CONFIG_IPIPE +static void ack_MSI_irq_w_maskbit(unsigned int vector) +{ + mask_MSI_irq(vector); + __ack_APIC_irq(); +} +static void ack_MSI_irq_wo_maskbit(unsigned int vector) +{ + __ack_APIC_irq(); +} +#else /* !CONFIG_IPIPE */ +#define ack_MSI_irq_wo_maskbit do_nothing +#define ack_MSI_irq_w_maskbit mask_MSI_irq +#endif /* CONFIG_IPIPE */ + static unsigned int startup_msi_irq_wo_maskbit(unsigned int vector) { struct msi_desc *entry; @@ -214,7 +229,7 @@ static struct hw_interrupt_type msix_irq .shutdown = shutdown_msi_irq, .enable = unmask_MSI_irq, .disable = mask_MSI_irq, - .ack = mask_MSI_irq, + .ack = ack_MSI_irq_w_maskbit, .end = end_msi_irq_w_maskbit, .set_affinity = set_msi_affinity }; @@ -230,7 +245,7 @@ static struct hw_interrupt_type msi_irq_ .shutdown = shutdown_msi_irq, .enable = unmask_MSI_irq, .disable = mask_MSI_irq, - .ack = mask_MSI_irq, + .ack = ack_MSI_irq_w_maskbit, .end = end_msi_irq_w_maskbit, .set_affinity = set_msi_affinity }; @@ -246,7 +261,7 @@ static struct hw_interrupt_type msi_irq_ .shutdown = shutdown_msi_irq, .enable = do_nothing, .disable = do_nothing, - .ack = do_nothing, + .ack = ack_MSI_irq_wo_maskbit, .end = end_msi_irq_wo_maskbit, .set_affinity = set_msi_affinity }; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/entry-macro.S linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/entry-macro.S --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/entry-macro.S 2006-05-07 16:42:04.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/entry-macro.S 2006-07-16 15:01:24.000000000 +0200 @@ -22,7 +22,11 @@ teq \irqstat, #0 ldreq \irqstat, [\base, #(INTEGRATOR_HDR_IC_OFFSET+IRQ_STATUS)] moveq \irqnr, #IRQ_CIC_START - +#ifdef CONFIG_IPIPE + tst \irqstat, #0x00000040 @ check IRQ_TIMERINT1 first + movne \irqnr, #6 + bne 1003f +#endif /* CONFIG_IPIPE */ 1001: tst \irqstat, #15 bne 1002f add \irqnr, \irqnr, #4 diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/platform.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/platform.h --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/platform.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/platform.h 2006-07-16 15:01:24.000000000 +0200 @@ -26,13 +26,15 @@ * NOTE: This is a multi-hosted header file for use with uHAL and * supported debuggers. * - * $Id: platform.s,v 1.32 2000/02/18 10:51:39 asims Exp $ + * $Id: platform.h,v 1.2 2006/02/20 13:54:22 rpm Exp $ * * ***********************************************************************/ #ifndef __address_h #define __address_h 1 +#include <linux/config.h> + /* ======================================================================== * Integrator definitions * ======================================================================== @@ -436,7 +438,7 @@ * Timer definitions * * Only use timer 1 & 2 - * (both run at 24MHz and will need the clock divider set to 16). + * (both run at 1MHZ on /CP and at 24MHz on /AP) * * Timer 0 runs at bus frequency and therefore could vary and currently * uHAL can't handle that. @@ -449,7 +451,12 @@ #define MAX_TIMER 2 #define MAX_PERIOD 699050 + +#ifdef CONFIG_ARCH_INTEGRATOR_CP +#define TICKS_PER_uSEC 1 +#else #define TICKS_PER_uSEC 24 +#endif /* * These are useconds NOT ticks. diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/timex.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/timex.h --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/timex.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/timex.h 2006-07-16 15:01:24.000000000 +0200 @@ -21,6 +21,6 @@ */ /* - * ?? + * Timer rate */ -#define CLOCK_TICK_RATE (50000000 / 16) +#define CLOCK_TICK_RATE (1000000) diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/atomic.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/atomic.h --- linux-2.6.16.5-tcl1/include/asm-arm/atomic.h 2006-05-07 16:42:05.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/atomic.h 2006-07-16 15:01:24.000000000 +0200 @@ -129,10 +129,10 @@ static inline int atomic_add_return(int unsigned long flags; int val; - local_irq_save(flags); + local_irq_save_hw(flags); val = v->counter; v->counter = val += i; - local_irq_restore(flags); + local_irq_restore_hw(flags); return val; } @@ -142,10 +142,10 @@ static inline int atomic_sub_return(int unsigned long flags; int val; - local_irq_save(flags); + local_irq_save_hw(flags); val = v->counter; v->counter = val -= i; - local_irq_restore(flags); + local_irq_restore_hw(flags); return val; } @@ -155,11 +155,11 @@ static inline int atomic_cmpxchg(atomic_ int ret; unsigned long flags; - local_irq_save(flags); + local_irq_save_hw(flags); ret = v->counter; if (likely(ret == old)) v->counter = new; - local_irq_restore(flags); + local_irq_restore_hw(flags); return ret; } @@ -168,9 +168,9 @@ static inline void atomic_clear_mask(uns { unsigned long flags; - local_irq_save(flags); + local_irq_save_hw(flags); *addr &= ~mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); } #endif /* __LINUX_ARM_ARCH__ */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/bitops.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/bitops.h --- linux-2.6.16.5-tcl1/include/asm-arm/bitops.h 2006-05-07 16:42:05.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/bitops.h 2006-07-16 15:01:24.000000000 +0200 @@ -37,9 +37,9 @@ static inline void ____atomic_set_bit(un p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); *p |= mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); } static inline void ____atomic_clear_bit(unsigned int bit, volatile unsigned long *p) @@ -49,9 +49,9 @@ static inline void ____atomic_clear_bit( p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); *p &= ~mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); } static inline void ____atomic_change_bit(unsigned int bit, volatile unsigned long *p) @@ -61,9 +61,9 @@ static inline void ____atomic_change_bit p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); *p ^= mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); } static inline int @@ -75,10 +75,10 @@ ____atomic_test_and_set_bit(unsigned int p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); res = *p; *p = res | mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); return res & mask; } @@ -92,10 +92,10 @@ ____atomic_test_and_clear_bit(unsigned i p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); res = *p; *p = res & ~mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); return res & mask; } @@ -109,10 +109,10 @@ ____atomic_test_and_change_bit(unsigned p += bit >> 5; - local_irq_save(flags); + local_irq_save_hw(flags); res = *p; *p = res ^ mask; - local_irq_restore(flags); + local_irq_restore_hw(flags); return res & mask; } diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h --- linux-2.6.16.5-tcl1/include/asm-arm/ipipe.h 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,193 @@ +/* -*- linux-c -*- + * include/asm-arm/ipipe.h + * + * Copyright (C) 2002-2005 Philippe Gerum. + * Copyright (C) 2005 Stelian Pop. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + */ + +#ifndef __ARM_IPIPE_H +#define __ARM_IPIPE_H + +#include <linux/config.h> + +#ifdef CONFIG_IPIPE + +#include <linux/list.h> +#include <linux/cpumask.h> +#include <linux/threads.h> +#include <asm/ptrace.h> +#include <asm/irq.h> +#include <asm/bitops.h> +#include <asm/mach/irq.h> + +#define IPIPE_ARCH_STRING "1.3-04" +#define IPIPE_MAJOR_NUMBER 1 +#define IPIPE_MINOR_NUMBER 3 +#define IPIPE_PATCH_NUMBER 4 + +#define IPIPE_NR_XIRQS NR_IRQS +#define IPIPE_IRQ_ISHIFT 5 /* 25 for 32bits arch. */ + +#ifdef CONFIG_SMP +#error "I-pipe/arm: SMP not yet implemented" +#define ipipe_processor_id() (current_thread_info()->cpu) +#else /* !CONFIG_SMP */ +#define ipipe_processor_id() 0 +#endif /* CONFIG_SMP */ + +/* Note that we disable the interrupts around context_switch, + * or we'll get into severe problems when scheduling Xenomaï + * user space real time threads. + * This can however cause high latencies, see for example: + * http://www.ussg.iu.edu/hypermail/linux/kernel/0405.2/1388.html + * This may need further optimization... + */ +#define prepare_arch_switch(next) \ +do { \ + __ipipe_dispatch_event(IPIPE_EVENT_SCHEDULE,next); \ + local_irq_disable_hw(); \ +} while(0) + +#define task_hijacked(p) \ + ( { \ + int x = ipipe_current_domain != ipipe_root_domain; \ + __clear_bit(IPIPE_SYNC_FLAG, \ + &ipipe_root_domain->cpudata[task_cpu(p)].status); \ + local_irq_enable_hw(); \ + x; \ + } ) + +/* ARM traps */ +#define IPIPE_TRAP_ACCESS 0 /* Data or instruction access exception */ +#define IPIPE_TRAP_SECTION 1 /* Section fault */ +#define IPIPE_TRAP_DABT 2 /* Generic data abort */ +#define IPIPE_TRAP_UNKNOWN 3 /* Unknown exception */ +#define IPIPE_TRAP_BREAK 4 /* Instruction breakpoint */ +#define IPIPE_TRAP_FPU 5 /* Floating point exception */ +#define IPIPE_TRAP_VFP 6 /* VFP floating point exception */ +#define IPIPE_TRAP_UNDEFINSTR 7 /* Undefined instruction */ +#define IPIPE_NR_FAULTS 8 + +/* Pseudo-vectors used for kernel events */ +#define IPIPE_FIRST_EVENT IPIPE_NR_FAULTS +#define IPIPE_EVENT_SYSCALL (IPIPE_FIRST_EVENT) +#define IPIPE_EVENT_SCHEDULE (IPIPE_FIRST_EVENT + 1) +#define IPIPE_EVENT_SIGWAKE (IPIPE_FIRST_EVENT + 2) +#define IPIPE_EVENT_SETSCHED (IPIPE_FIRST_EVENT + 3) +#define IPIPE_EVENT_EXIT (IPIPE_FIRST_EVENT + 4) +#define IPIPE_EVENT_CLEANUP (IPIPE_FIRST_EVENT + 5) +#define IPIPE_LAST_EVENT IPIPE_EVENT_CLEANUP +#define IPIPE_NR_EVENTS (IPIPE_LAST_EVENT + 1) + +struct ipipe_domain; + +struct ipipe_sysinfo { + + int ncpus; /* Number of CPUs on board */ + u64 cpufreq; /* CPU frequency (in Hz) */ + + /* Arch-dependent block */ + + struct { + unsigned tmirq; /* Timer tick IRQ */ + u64 tmfreq; /* Timer frequency */ + } archdep; +}; + +/* arch specific stuff */ +extern int __ipipe_mach_timerint; +extern int __ipipe_mach_timerstolen; +extern unsigned int __ipipe_mach_ticks_per_jiffy; +extern void __ipipe_mach_acktimer(void); +extern unsigned long long __ipipe_mach_get_tsc(void); +extern void __ipipe_mach_set_dec(unsigned long); +extern unsigned long __ipipe_mach_get_dec(void); + +#define ipipe_read_tsc(t) do { t = __ipipe_mach_get_tsc(); } while (0) +#define __ipipe_read_timebase() __ipipe_mach_get_tsc() + +#define ipipe_cpu_freq() (HZ * __ipipe_mach_ticks_per_jiffy) +#define ipipe_tsc2ns(t) (((t) * 1000) / (ipipe_cpu_freq() / 1000000)) + +/* Private interface -- Internal use only */ + +#define __ipipe_check_platform() do { } while(0) + +#define __ipipe_enable_irq(irq) irq_desc[irq].chip->unmask(irq) + +#define __ipipe_disable_irq(irq) irq_desc[irq].chip->mask(irq) + +void __ipipe_init_platform(void); + +void __ipipe_enable_pipeline(void); + +int __ipipe_ack_irq(unsigned irq); + +int __ipipe_ack_timerirq(unsigned irq); + +void __ipipe_do_IRQ(int irq, + struct pt_regs *regs); + +void __ipipe_do_timer(int irq, + struct pt_regs *regs); + +void __ipipe_do_critical_sync(unsigned irq, + void *cookie); + +extern unsigned long __ipipe_decr_ticks; + +extern unsigned long long __ipipe_decr_next[]; + +extern struct pt_regs __ipipe_tick_regs[]; + +void __ipipe_handle_irq(int irq, + struct pt_regs *regs); + +#define __ipipe_tick_irq ipipe_timerint + +static inline unsigned long __ipipe_ffnz(unsigned long ul) +{ + return ffs(ul) - 1; +} + +#define __ipipe_run_isr(ipd, irq, cpuid) \ +do { \ + if (ipd == ipipe_root_domain) { \ + /* \ + * Linux handlers are called w/ hw interrupts on so \ + * that they could not defer interrupts for higher \ + * priority domains. \ + */ \ + local_irq_enable_hw(); \ + ((void (*)(unsigned, struct pt_regs *)) \ + ipd->irqs[irq].handler) (irq, __ipipe_tick_regs + cpuid); \ + local_irq_disable_hw(); \ + } else { \ + __clear_bit(IPIPE_SYNC_FLAG, &cpudata->status); \ + ipd->irqs[irq].handler(irq,ipd->irqs[irq].cookie); \ + __set_bit(IPIPE_SYNC_FLAG, &cpudata->status); \ + } \ +} while(0) + +#else /* !CONFIG_IPIPE */ + +#define task_hijacked(p) 0 + +#endif /* CONFIG_IPIPE */ + +#endif /* !__ARM_IPIPE_H */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/mmu_context.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/mmu_context.h --- linux-2.6.16.5-tcl1/include/asm-arm/mmu_context.h 2006-05-07 15:36:58.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/mmu_context.h 2006-07-16 15:01:24.000000000 +0200 @@ -82,14 +82,17 @@ static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next, struct task_struct *tsk) { - unsigned int cpu = smp_processor_id(); + unsigned int cpu = smp_processor_id_hw(); if (prev != next) { + unsigned long flags; + local_irq_save_hw_cond(flags); cpu_set(cpu, next->cpu_vm_mask); check_context(next); cpu_switch_mm(next->pgd, next); if (cache_is_vivt()) cpu_clear(cpu, prev->cpu_vm_mask); + local_irq_restore_hw_cond(flags); } } diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/pgalloc.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/pgalloc.h --- linux-2.6.16.5-tcl1/include/asm-arm/pgalloc.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/pgalloc.h 2006-07-16 15:01:24.000000000 +0200 @@ -29,6 +29,11 @@ extern void free_pgd_slow(pgd_t *pgd); #define check_pgt_cache() do { } while (0) +static inline void set_pgdir(unsigned long address, pgd_t entry) +{ + /* nop */ +} + /* * Allocate one PTE table. * diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/system.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h --- linux-2.6.16.5-tcl1/include/asm-arm/system.h 2006-05-07 16:42:05.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h 2006-07-17 18:40:59.000000000 +0200 @@ -186,30 +186,30 @@ static inline void sched_cacheflush(void */ #if __LINUX_ARM_ARCH__ >= 6 -#define local_irq_save(x) \ +#define local_irq_save_hw(x) \ ({ \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_irq_save\n" \ + "mrs %0, cpsr @ local_irq_save_hw\n" \ "cpsid i" \ : "=r" (x) : : "memory", "cc"); \ }) -#define local_irq_enable() __asm__("cpsie i @ __sti" : : : "memory", "cc") -#define local_irq_disable() __asm__("cpsid i @ __cli" : : : "memory", "cc") -#define local_fiq_enable() __asm__("cpsie f @ __stf" : : : "memory", "cc") -#define local_fiq_disable() __asm__("cpsid f @ __clf" : : : "memory", "cc") +#define local_irq_enable_hw() __asm__("cpsie i @ __sti" : : : "memory", "cc") +#define local_irq_disable_hw() __asm__("cpsid i @ __cli" : : : "memory", "cc") +#define local_fiq_enable_hw() __asm__("cpsie f @ __stf" : : : "memory", "cc") +#define local_fiq_disable_hw() __asm__("cpsid f @ __clf" : : : "memory", "cc") #else /* * Save the current interrupt enable state & disable IRQs */ -#define local_irq_save(x) \ +#define local_irq_save_hw(x) \ ({ \ unsigned long temp; \ (void) (&temp == &x); \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_irq_save\n" \ + "mrs %0, cpsr @ local_irq_save_hw\n" \ " orr %1, %0, #128\n" \ " msr cpsr_c, %1" \ : "=r" (x), "=r" (temp) \ @@ -220,11 +220,11 @@ static inline void sched_cacheflush(void /* * Enable IRQs */ -#define local_irq_enable() \ +#define local_irq_enable_hw() \ ({ \ unsigned long temp; \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_irq_enable\n" \ + "mrs %0, cpsr @ local_irq_enable_hw\n"\ " bic %0, %0, #128\n" \ " msr cpsr_c, %0" \ : "=r" (temp) \ @@ -235,11 +235,11 @@ static inline void sched_cacheflush(void /* * Disable IRQs */ -#define local_irq_disable() \ +#define local_irq_disable_hw() \ ({ \ unsigned long temp; \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_irq_disable\n" \ + "mrs %0, cpsr @ local_irq_disable_hw\n"\ " orr %0, %0, #128\n" \ " msr cpsr_c, %0" \ : "=r" (temp) \ @@ -250,7 +250,7 @@ static inline void sched_cacheflush(void /* * Enable FIQs */ -#define local_fiq_enable() \ +#define local_fiq_enable_hw() \ ({ \ unsigned long temp; \ __asm__ __volatile__( \ @@ -265,7 +265,7 @@ static inline void sched_cacheflush(void /* * Disable FIQs */ -#define local_fiq_disable() \ +#define local_fiq_disable_hw() \ ({ \ unsigned long temp; \ __asm__ __volatile__( \ @@ -282,29 +282,63 @@ static inline void sched_cacheflush(void /* * Save the current interrupt enable state. */ -#define local_save_flags(x) \ +#define local_save_flags_hw(x) \ ({ \ __asm__ __volatile__( \ - "mrs %0, cpsr @ local_save_flags" \ + "mrs %0, cpsr @ local_save_flags_hw" \ : "=r" (x) : : "memory", "cc"); \ }) /* * restore saved IRQ & FIQ state */ -#define local_irq_restore(x) \ +#define local_irq_restore_hw(x) \ __asm__ __volatile__( \ - "msr cpsr_c, %0 @ local_irq_restore\n" \ + "msr cpsr_c, %0 @ local_irq_restore_hw\n"\ : \ : "r" (x) \ : "memory", "cc") -#define irqs_disabled() \ -({ \ +#define irqs_disabled_hw() \ + ({ \ unsigned long flags; \ - local_save_flags(flags); \ + local_save_flags_hw(flags); \ (int)(flags & PSR_I_BIT); \ -}) + }) + + +#ifdef CONFIG_IPIPE + +void __ipipe_stall_root(void); +void __ipipe_unstall_root(void); +unsigned long __ipipe_test_root(void); +unsigned long __ipipe_test_and_stall_root(void); +void __ipipe_restore_root(unsigned long flags); + +/* PSR_I_BIT is bit no. 7 and is set if interrupts are _disabled_ */ +#define local_irq_save(flags) ((flags) = __ipipe_test_and_stall_root() << 7) +#define local_irq_enable() __ipipe_unstall_root() +#define local_irq_disable() __ipipe_stall_root() +#define local_fiq_enable() __ipipe_unstall_root() +#define local_fiq_disable() __ipipe_stall_root() +#define local_save_flags(flags) ((flags) = __ipipe_test_root() << 7) +#define local_irq_restore(flags) __ipipe_restore_root(flags & (1 << 7)) + +#define irqs_disabled() __ipipe_test_root() + +#else /* !CONFIG_IPIPE */ + +#define local_irq_save(flags) local_irq_save_hw(flags) +#define local_irq_enable() local_irq_enable_hw() +#define local_irq_disable() local_irq_disable_hw() +#define local_fiq_enable() local_fiq_enable_hw() +#define local_fiq_disable() local_fiq_disable_hw() +#define local_save_flags(flags) local_save_flags_hw(flags) +#define local_irq_restore(flags) local_irq_restore_hw(flags) + +#define irqs_disabled() irqs_disabled_hw() + +#endif /* CONFIG_IPIPE */ #ifdef CONFIG_SMP @@ -379,17 +413,17 @@ static inline unsigned long __xchg(unsig #error SMP is not supported on this platform #endif case 1: - local_irq_save(flags); + local_irq_save_hw(flags); ret = *(volatile unsigned char *)ptr; *(volatile unsigned char *)ptr = x; - local_irq_restore(flags); + local_irq_restore_hw(flags); break; case 4: - local_irq_save(flags); + local_irq_save_hw(flags); ret = *(volatile unsigned long *)ptr; *(volatile unsigned long *)ptr = x; - local_irq_restore(flags); + local_irq_restore_hw(flags); break; #else case 1: diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/hardirq.h linux-2.6.16.5-tcl1-ipipe/include/linux/hardirq.h --- linux-2.6.16.5-tcl1/include/linux/hardirq.h 2006-05-07 16:42:11.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/hardirq.h 2006-07-16 15:01:24.000000000 +0200 @@ -87,8 +87,21 @@ extern void synchronize_irq(unsigned int # define synchronize_irq(irq) barrier() #endif +#ifdef CONFIG_IPIPE +#define nmi_enter() \ +do { \ + if (ipipe_current_domain == ipipe_root_domain) \ + irq_enter(); \ +} while(0) +#define nmi_exit() \ +do { \ + if (ipipe_current_domain == ipipe_root_domain) \ + sub_preempt_count(HARDIRQ_OFFSET); \ +} while(0) +#else /* !CONFIG_IPIPE */ #define nmi_enter() irq_enter() #define nmi_exit() sub_preempt_count(HARDIRQ_OFFSET) +#endif /* CONFIG_IPIPE */ struct task_struct; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/linux/ipipe.h --- linux-2.6.16.5-tcl1/include/linux/ipipe.h 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/ipipe.h 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,699 @@ +/* -*- linux-c -*- + * include/linux/ipipe.h + * + * Copyright (C) 2002-2005 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + */ + +#ifndef __LINUX_IPIPE_H +#define __LINUX_IPIPE_H + +#include <linux/config.h> +#include <linux/spinlock.h> +#include <linux/cache.h> +#include <asm/ipipe.h> + +#ifdef CONFIG_IPIPE + +#define IPIPE_VERSION_STRING IPIPE_ARCH_STRING +#define IPIPE_RELEASE_NUMBER ((IPIPE_MAJOR_NUMBER << 16) | \ + (IPIPE_MINOR_NUMBER << 8) | \ + (IPIPE_PATCH_NUMBER)) + +#ifndef BROKEN_BUILTIN_RETURN_ADDRESS +#define __BUILTIN_RETURN_ADDRESS0 ((unsigned long)__builtin_return_address(0)) +#define __BUILTIN_RETURN_ADDRESS1 ((unsigned long)__builtin_return_address(1)) +#endif /* !BUILTIN_RETURN_ADDRESS */ + +#define IPIPE_ROOT_PRIO 100 +#define IPIPE_ROOT_ID 0 +#define IPIPE_ROOT_NPTDKEYS 4 /* Must be <= BITS_PER_LONG */ + +#define IPIPE_RESET_TIMER 0x1 +#define IPIPE_GRAB_TIMER 0x2 + +/* Global domain flags */ +#define IPIPE_SPRINTK_FLAG 0 /* Synchronous printk() allowed */ +#define IPIPE_AHEAD_FLAG 1 /* Domain always heads the pipeline */ + +#define IPIPE_STALL_FLAG 0 /* Stalls a pipeline stage -- guaranteed at bit #0 */ +#define IPIPE_SYNC_FLAG 1 /* The interrupt syncer is running for the domain */ + +#define IPIPE_SYNC_MASK (1 << IPIPE_SYNC_FLAG) + +#define IPIPE_HANDLE_FLAG 0 +#define IPIPE_PASS_FLAG 1 +#define IPIPE_ENABLE_FLAG 2 +#define IPIPE_DYNAMIC_FLAG IPIPE_HANDLE_FLAG +#define IPIPE_STICKY_FLAG 3 +#define IPIPE_SYSTEM_FLAG 4 +#define IPIPE_LOCK_FLAG 5 +#define IPIPE_SHARED_FLAG 6 +#define IPIPE_WIRED_FLAG 7 +#define IPIPE_EXCLUSIVE_FLAG 8 + +#define IPIPE_HANDLE_MASK (1 << IPIPE_HANDLE_FLAG) +#define IPIPE_PASS_MASK (1 << IPIPE_PASS_FLAG) +#define IPIPE_ENABLE_MASK (1 << IPIPE_ENABLE_FLAG) +#define IPIPE_DYNAMIC_MASK IPIPE_HANDLE_MASK +#define IPIPE_STICKY_MASK (1 << IPIPE_STICKY_FLAG) +#define IPIPE_SYSTEM_MASK (1 << IPIPE_SYSTEM_FLAG) +#define IPIPE_LOCK_MASK (1 << IPIPE_LOCK_FLAG) +#define IPIPE_SHARED_MASK (1 << IPIPE_SHARED_FLAG) +#define IPIPE_WIRED_MASK (1 << IPIPE_WIRED_FLAG) +#define IPIPE_EXCLUSIVE_MASK (1 << IPIPE_EXCLUSIVE_FLAG) + +#define IPIPE_DEFAULT_MASK (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK) +#define IPIPE_STDROOT_MASK (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK|IPIPE_SYSTEM_MASK) + +#define IPIPE_EVENT_SELF 0x80000000 + +/* Number of virtual IRQs */ +#define IPIPE_NR_VIRQS BITS_PER_LONG +/* First virtual IRQ # */ +#define IPIPE_VIRQ_BASE (((IPIPE_NR_XIRQS + BITS_PER_LONG - 1) / BITS_PER_LONG) * BITS_PER_LONG) +/* Total number of IRQ slots */ +#define IPIPE_NR_IRQS (IPIPE_VIRQ_BASE + IPIPE_NR_VIRQS) +/* Number of indirect words needed to map the whole IRQ space. */ +#define IPIPE_IRQ_IWORDS ((IPIPE_NR_IRQS + BITS_PER_LONG - 1) / BITS_PER_LONG) +#define IPIPE_IRQ_IMASK (BITS_PER_LONG - 1) +#define IPIPE_IRQMASK_ANY (~0L) +#define IPIPE_IRQMASK_VIRT (IPIPE_IRQMASK_ANY << (IPIPE_VIRQ_BASE / BITS_PER_LONG)) + +#ifdef CONFIG_SMP + +#define IPIPE_NR_CPUS NR_CPUS +#define ipipe_declare_cpuid int cpuid +#define ipipe_load_cpuid() do { \ + (cpuid) = ipipe_processor_id(); \ + } while(0) +#define ipipe_lock_cpu(flags) do { \ + local_irq_save_hw(flags); \ + (cpuid) = ipipe_processor_id(); \ + } while(0) +#define ipipe_unlock_cpu(flags) local_irq_restore_hw(flags) +#define ipipe_get_cpu(flags) ipipe_lock_cpu(flags) +#define ipipe_put_cpu(flags) ipipe_unlock_cpu(flags) +#define ipipe_current_domain (ipipe_percpu_domain[ipipe_processor_id()]) + +#else /* !CONFIG_SMP */ + +#define IPIPE_NR_CPUS 1 +#define ipipe_declare_cpuid const int cpuid = 0 +#define ipipe_load_cpuid() do { } while(0) +#define ipipe_lock_cpu(flags) local_irq_save_hw(flags) +#define ipipe_unlock_cpu(flags) local_irq_restore_hw(flags) +#define ipipe_get_cpu(flags) do { (void)(flags); } while(0) +#define ipipe_put_cpu(flags) do { } while(0) +#define ipipe_current_domain (ipipe_percpu_domain[0]) + +#endif /* CONFIG_SMP */ + +#define ipipe_virtual_irq_p(irq) ((irq) >= IPIPE_VIRQ_BASE && \ + (irq) < IPIPE_NR_IRQS) + +typedef void (*ipipe_irq_handler_t)(unsigned irq, + void *cookie); + +#define IPIPE_SAME_HANDLER ((ipipe_irq_handler_t)(-1)) + +typedef int (*ipipe_irq_ackfn_t)(unsigned irq); + +typedef int (*ipipe_event_handler_t)(unsigned event, + struct ipipe_domain *from, + void *data); +struct ipipe_domain { + + struct list_head p_link; /* Link in pipeline */ + + struct ipcpudata { + unsigned long status; + unsigned long irq_pending_hi; + unsigned long irq_pending_lo[IPIPE_IRQ_IWORDS]; + struct ipirqcnt { + unsigned long pending_hits; + unsigned long total_hits; + } irq_counters[IPIPE_NR_IRQS]; + } ____cacheline_aligned_in_smp cpudata[IPIPE_NR_CPUS]; + + struct { + unsigned long control; + ipipe_irq_ackfn_t acknowledge; + ipipe_irq_handler_t handler; + void *cookie; + } ____cacheline_aligned irqs[IPIPE_NR_IRQS]; + + ipipe_event_handler_t evhand[IPIPE_NR_EVENTS]; /* Event handlers. */ + unsigned long long evself; /* Self-monitored event bits. */ + unsigned long flags; + unsigned domid; + const char *name; + int priority; + void *pdd; +}; + +#define IPIPE_HEAD_PRIORITY (-1) /* For domains always heading the pipeline */ + +struct ipipe_domain_attr { + + unsigned domid; /* Domain identifier -- Magic value set by caller */ + const char *name; /* Domain name -- Warning: won't be dup'ed! */ + int priority; /* Priority in interrupt pipeline */ + void (*entry) (void); /* Domain entry point */ + void *pdd; /* Per-domain (opaque) data pointer */ +}; + +/* The following macros must be used hw interrupts off. */ + +#define __ipipe_irq_cookie(ipd,irq) (ipd)->irqs[irq].cookie +#define __ipipe_irq_handler(ipd,irq) (ipd)->irqs[irq].handler + +#define __ipipe_cpudata_irq_hits(ipd,cpuid,irq) ((ipd)->cpudata[cpuid].irq_counters[irq].total_hits) + +#define __ipipe_set_irq_bit(ipd,cpuid,irq) \ +do { \ + if (!test_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) { \ + __set_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + __set_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[cpuid].irq_pending_hi); \ + } \ +} while(0) + +#define __ipipe_clear_pend(ipd,cpuid,irq) \ +do { \ + __clear_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + if ((ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT] == 0) \ + __clear_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[cpuid].irq_pending_hi); \ +} while(0) + +#define __ipipe_lock_irq(ipd,cpuid,irq) \ +do { \ + if (!test_and_set_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) \ + __ipipe_clear_pend(ipd,cpuid,irq); \ +} while(0) + +#define __ipipe_unlock_irq(ipd,irq) \ +do { \ + int __cpuid, __nr_cpus = num_online_cpus(); \ + if (test_and_clear_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) \ + for (__cpuid = 0; __cpuid < __nr_cpus; __cpuid++) \ + if ((ipd)->cpudata[__cpuid].irq_counters[irq].pending_hits > 0) { /* We need atomic ops next. */ \ + set_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[__cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + set_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[__cpuid].irq_pending_hi); \ + } \ +} while(0) + +#define __ipipe_clear_irq(ipd,irq) \ +do { \ + int __cpuid, __nr_cpus = num_online_cpus(); \ + clear_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control); \ + for (__cpuid = 0; __cpuid < __nr_cpus; __cpuid++) { \ + (ipd)->cpudata[__cpuid].irq_counters[irq].pending_hits = 0; \ + __ipipe_clear_pend(ipd,__cpuid,irq); \ + } \ +} while(0) + +#ifdef __RAW_SPIN_LOCK_UNLOCKED +#define spin_lock_hw(x) _raw_spin_lock(x) +#define spin_trylock_hw(x) _raw_spin_trylock(x) +#define spin_unlock_hw(x) _raw_spin_unlock(x) +#if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK) +#define write_lock_hw(x) _raw_write_lock(x) +#define write_trylock_hw(x) _raw_write_trylock(x) +#define write_unlock_hw(x) _raw_write_unlock(x) +#define read_lock_hw(x) _raw_read_lock(x) +#define read_trylock_hw(x) _raw_read_trylock(x) +#define read_unlock_hw(x) _raw_read_unlock(x) +#else /* UP non-debug */ +#define write_lock_hw(lock) do { (void)(lock); } while (0) +#define write_trylock_hw(lock) ({ (void)(lock); 1; }) +#define write_unlock_hw(lock) do { (void)(lock); } while (0) +#define read_lock_hw(lock) do { (void)(lock); } while (0) +#define read_trylock_hw(lock) ({ (void)(lock); 1; }) +#define read_unlock_hw(lock) do { (void)(lock); } while (0) +#endif /* CONFIG_SMP || CONFIG_DEBUG_SPINLOCK */ +#else /* !__RAW_SPIN_LOCK_UNLOCKED */ +#define spin_lock_hw(x) _spin_lock(x) +#define spin_unlock_hw(x) _spin_unlock(x) +#define spin_trylock_hw(x) _spin_trylock(x) +#define write_lock_hw(x) _write_lock(x) +#define write_unlock_hw(x) _write_unlock(x) +#define write_trylock_hw(x) _write_trylock(x) +#define read_lock_hw(x) _read_lock(x) +#define read_unlock_hw(x) _read_unlock(x) +#endif /* __RAW_SPIN_LOCK_UNLOCKED */ + +typedef spinlock_t ipipe_spinlock_t; +typedef rwlock_t ipipe_rwlock_t; +#define IPIPE_SPIN_LOCK_UNLOCKED SPIN_LOCK_UNLOCKED +#define IPIPE_RW_LOCK_UNLOCKED RW_LOCK_UNLOCKED + +#define spin_lock_irqsave_hw(x,flags) \ +do { \ + local_irq_save_hw(flags); \ + spin_lock_hw(x); \ +} while (0) + +#define spin_unlock_irqrestore_hw(x,flags) \ +do { \ + spin_unlock_hw(x); \ + local_irq_restore_hw(flags); \ +} while (0) + +#define spin_lock_irq_hw(x) \ +do { \ + local_irq_disable_hw(); \ + spin_lock_hw(x); \ +} while (0) + +#define spin_unlock_irq_hw(x) \ +do { \ + spin_unlock_hw(x); \ + local_irq_enable_hw(); \ +} while (0) + +#define read_lock_irqsave_hw(lock, flags) \ +do { \ + local_irq_save_hw(flags); \ + read_lock_hw(lock); \ +} while (0) + +#define read_unlock_irqrestore_hw(lock, flags) \ +do { \ + read_unlock_hw(lock); \ + local_irq_restore_hw(flags); \ +} while (0) + +#define write_lock_irqsave_hw(lock, flags) \ +do { \ + local_irq_save_hw(flags); \ + write_lock_hw(lock); \ +} while (0) + +#define write_unlock_irqrestore_hw(lock, flags) \ +do { \ + write_unlock_hw(lock); \ + local_irq_restore_hw(flags); \ +} while (0) + +extern struct ipipe_domain *ipipe_percpu_domain[], *ipipe_root_domain; + +extern unsigned __ipipe_printk_virq; + +extern unsigned long __ipipe_virtual_irq_map; + +extern struct list_head __ipipe_pipeline; + +extern ipipe_spinlock_t __ipipe_pipelock; + +extern int __ipipe_event_monitors[]; + +/* Private interface */ + +void ipipe_init(void); + +#ifdef CONFIG_PROC_FS +void ipipe_init_proc(void); + +#ifdef CONFIG_IPIPE_TRACE +void __ipipe_init_trace_proc(void); +#else /* !CONFIG_IPIPE_TRACE */ +#define __ipipe_init_trace_proc() do { } while(0) +#endif /* CONFIG_IPIPE_TRACE */ + +#else /* !CONFIG_PROC_FS */ +#define ipipe_init_proc() do { } while(0) +#endif /* CONFIG_PROC_FS */ + +void __ipipe_init_stage(struct ipipe_domain *ipd); + +void __ipipe_cleanup_domain(struct ipipe_domain *ipd); + +void __ipipe_add_domain_proc(struct ipipe_domain *ipd); + +void __ipipe_remove_domain_proc(struct ipipe_domain *ipd); + +void __ipipe_flush_printk(unsigned irq, void *cookie); + +void __ipipe_stall_root(void); + +void __ipipe_unstall_root(void); + +unsigned long __ipipe_test_root(void); + +unsigned long __ipipe_test_and_stall_root(void); + +void fastcall __ipipe_walk_pipeline(struct list_head *pos, int cpuid); + +void fastcall __ipipe_restore_root(unsigned long x); + +int fastcall __ipipe_schedule_irq(unsigned irq, struct list_head *head); + +int fastcall __ipipe_dispatch_event(unsigned event, void *data); + +int fastcall __ipipe_dispatch_wired(struct ipipe_domain *head, unsigned irq); + +void fastcall __ipipe_sync_stage(unsigned long syncmask); + +#ifndef __ipipe_sync_pipeline +#define __ipipe_sync_pipeline(syncmask) __ipipe_sync_stage(syncmask) +#endif + +#ifndef __ipipe_run_irqtail +#define __ipipe_run_irqtail() do { } while(0) +#endif + +#define __ipipe_pipeline_head_p(ipd) (&(ipd)->p_link == __ipipe_pipeline.next) + +/* + * Keep the following as a macro, so that client code could check for + * the support of the invariant pipeline head optimization. + */ +#define __ipipe_pipeline_head() list_entry(__ipipe_pipeline.next,struct ipipe_domain,p_link) + +#define __ipipe_event_pipelined_p(ev) \ + (__ipipe_event_monitors[ev] > 0 || (ipipe_current_domain->evself & (1LL << ev))) + +#ifdef CONFIG_SMP + +cpumask_t __ipipe_set_irq_affinity(unsigned irq, + cpumask_t cpumask); + +int fastcall __ipipe_send_ipi(unsigned ipi, + cpumask_t cpumask); + +#endif /* CONFIG_SMP */ + +/* Called with hw interrupts off. */ +static inline void __ipipe_switch_to(struct ipipe_domain *out, + struct ipipe_domain *in, int cpuid) +{ + void ipipe_suspend_domain(void); + + /* + * "in" is guaranteed to be closer than "out" from the head of the + * pipeline (and obviously different). + */ + + ipipe_percpu_domain[cpuid] = in; + + ipipe_suspend_domain(); /* Sync stage and propagate interrupts. */ + ipipe_load_cpuid(); /* Processor might have changed. */ + + if (ipipe_percpu_domain[cpuid] == in) + /* + * Otherwise, something has changed the current domain under + * our feet recycling the register set; do not override. + */ + ipipe_percpu_domain[cpuid] = out; +} + +static inline void ipipe_sigwake_notify(struct task_struct *p) +{ + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SIGWAKE)) + __ipipe_dispatch_event(IPIPE_EVENT_SIGWAKE,p); +} + +static inline void ipipe_setsched_notify(struct task_struct *p) +{ + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SETSCHED)) + __ipipe_dispatch_event(IPIPE_EVENT_SETSCHED,p); +} + +static inline void ipipe_exit_notify(struct task_struct *p) +{ + if (__ipipe_event_pipelined_p(IPIPE_EVENT_EXIT)) + __ipipe_dispatch_event(IPIPE_EVENT_EXIT,p); +} + +static inline int ipipe_trap_notify(int ex, struct pt_regs *regs) +{ + return __ipipe_event_pipelined_p(ex) ? __ipipe_dispatch_event(ex,regs) : 0; +} + +struct mm_struct; + +static inline void ipipe_cleanup_notify(struct mm_struct *mm) +{ + if (__ipipe_event_pipelined_p(IPIPE_EVENT_CLEANUP)) + __ipipe_dispatch_event(IPIPE_EVENT_CLEANUP,mm); +} + +/* Public interface */ + +int ipipe_register_domain(struct ipipe_domain *ipd, + struct ipipe_domain_attr *attr); + +int ipipe_unregister_domain(struct ipipe_domain *ipd); + +void ipipe_suspend_domain(void); + +int ipipe_virtualize_irq(struct ipipe_domain *ipd, + unsigned irq, + ipipe_irq_handler_t handler, + void *cookie, + ipipe_irq_ackfn_t acknowledge, + unsigned modemask); + +static inline int ipipe_share_irq(unsigned irq, + ipipe_irq_ackfn_t acknowledge) +{ + return ipipe_virtualize_irq(ipipe_current_domain, + irq, + IPIPE_SAME_HANDLER, + NULL, + acknowledge, + IPIPE_SHARED_MASK | IPIPE_HANDLE_MASK | + IPIPE_PASS_MASK); +} + +int ipipe_control_irq(unsigned irq, + unsigned clrmask, + unsigned setmask); + +unsigned ipipe_alloc_virq(void); + +int ipipe_free_virq(unsigned virq); + +int fastcall ipipe_trigger_irq(unsigned irq); + +static inline int ipipe_propagate_irq(unsigned irq) +{ + return __ipipe_schedule_irq(irq, ipipe_current_domain->p_link.next); +} + +static inline int ipipe_schedule_irq(unsigned irq) +{ + return __ipipe_schedule_irq(irq, &ipipe_current_domain->p_link); +} + +void fastcall ipipe_stall_pipeline_from(struct ipipe_domain *ipd); + +unsigned long fastcall ipipe_test_and_stall_pipeline_from(struct ipipe_domain *ipd); + +void fastcall ipipe_unstall_pipeline_from(struct ipipe_domain *ipd); + +unsigned long fastcall ipipe_test_and_unstall_pipeline_from(struct ipipe_domain *ipd); + +void fastcall ipipe_restore_pipeline_from(struct ipipe_domain *ipd, + unsigned long x); + +static inline unsigned long ipipe_test_pipeline_from(struct ipipe_domain *ipd) +{ + unsigned long flags, x; + ipipe_declare_cpuid; + + ipipe_get_cpu(flags); + x = test_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + ipipe_put_cpu(flags); + + return x; +} + +static inline void ipipe_restore_pipeline_nosync(struct ipipe_domain *ipd, + unsigned long x, int cpuid) +{ + /* + * If cpuid is current, then it must be held on entry + * (ipipe_get_cpu/local_irq_save_hw/local_irq_disable_hw). + */ + + if (x) + __set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + else + __clear_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); +} + +static inline void ipipe_stall_pipeline_head(void) +{ + ipipe_declare_cpuid; + unsigned long flags; + + ipipe_lock_cpu(flags); + __set_bit(IPIPE_STALL_FLAG, &__ipipe_pipeline_head()->cpudata[cpuid].status); +} + +static inline unsigned long ipipe_test_and_stall_pipeline_head(void) +{ + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + return __test_and_set_bit(IPIPE_STALL_FLAG, &__ipipe_pipeline_head()->cpudata[cpuid].status); +} + +void ipipe_unstall_pipeline_head(void); + +void fastcall __ipipe_restore_pipeline_head(struct ipipe_domain *head, + unsigned long x); + +static inline void ipipe_restore_pipeline_head(unsigned long x) +{ + struct ipipe_domain *head = __ipipe_pipeline_head(); + /* On some archs, __test_and_set_bit() might return different + * truth value than test_bit(), so we test the exclusive OR of + * both statuses, assuming that the lowest bit is always set in + * the truth value (if this is wrong, the failed optimization will + * be caught in __ipipe_restore_pipeline_head() if + * CONFIG_DEBUG_KERNEL is set). */ + if ((x ^ test_bit(IPIPE_STALL_FLAG, &head->cpudata[ipipe_processor_id()].status)) & 1) + __ipipe_restore_pipeline_head(head,x); +} + +#define ipipe_unstall_pipeline() \ + ipipe_unstall_pipeline_from(ipipe_current_domain) + +#define ipipe_test_and_unstall_pipeline() \ + ipipe_test_and_unstall_pipeline_from(ipipe_current_domain) + +#define ipipe_test_pipeline() \ + ipipe_test_pipeline_from(ipipe_current_domain) + +#define ipipe_test_and_stall_pipeline() \ + ipipe_test_and_stall_pipeline_from(ipipe_current_domain) + +#define ipipe_stall_pipeline() \ + ipipe_stall_pipeline_from(ipipe_current_domain) + +#define ipipe_restore_pipeline(x) \ + ipipe_restore_pipeline_from(ipipe_current_domain, (x)) + +void ipipe_init_attr(struct ipipe_domain_attr *attr); + +int ipipe_get_sysinfo(struct ipipe_sysinfo *sysinfo); + +int ipipe_tune_timer(unsigned long ns, + int flags); + +unsigned long ipipe_critical_enter(void (*syncfn) (void)); + +void ipipe_critical_exit(unsigned long flags); + +static inline void ipipe_set_printk_sync(struct ipipe_domain *ipd) +{ + set_bit(IPIPE_SPRINTK_FLAG, &ipd->flags); +} + +static inline void ipipe_set_printk_async(struct ipipe_domain *ipd) +{ + clear_bit(IPIPE_SPRINTK_FLAG, &ipd->flags); +} + +ipipe_event_handler_t ipipe_catch_event(struct ipipe_domain *ipd, + unsigned event, + ipipe_event_handler_t handler); + +cpumask_t ipipe_set_irq_affinity(unsigned irq, + cpumask_t cpumask); + +int fastcall ipipe_send_ipi(unsigned ipi, + cpumask_t cpumask); + +int ipipe_setscheduler_root(struct task_struct *p, + int policy, + int prio); + +int ipipe_reenter_root(struct task_struct *prev, + int policy, + int prio); + +int ipipe_alloc_ptdkey(void); + +int ipipe_free_ptdkey(int key); + +int fastcall ipipe_set_ptd(int key, + void *value); + +void fastcall *ipipe_get_ptd(int key); + +#define local_irq_enable_hw_cond() local_irq_enable_hw() +#define local_irq_disable_hw_cond() local_irq_disable_hw() +#define local_irq_save_hw_cond(flags) local_irq_save_hw(flags) +#define local_irq_restore_hw_cond(flags) local_irq_restore_hw(flags) +#define spin_lock_irqsave_hw_cond(lock,flags) spin_lock_irqsave_hw(lock,flags) +#define spin_unlock_irqrestore_hw_cond(lock,flags) spin_unlock_irqrestore_hw(lock,flags) +#define smp_processor_id_hw() ipipe_processor_id() + +#define ipipe_irq_lock(irq) \ + do { \ + ipipe_declare_cpuid; \ + ipipe_load_cpuid(); \ + __ipipe_lock_irq(ipipe_percpu_domain[cpuid], cpuid, irq);\ + } while(0) + +#define ipipe_irq_unlock(irq) \ + do { \ + ipipe_declare_cpuid; \ + ipipe_load_cpuid(); \ + __ipipe_unlock_irq(ipipe_percpu_domain[cpuid], irq); \ + } while(0) + +#define ipipe_root_domain_p (ipipe_current_domain == ipipe_root_domain) + +#else /* !CONFIG_IPIPE */ + +#define ipipe_init() do { } while(0) +#define ipipe_suspend_domain() do { } while(0) +#define ipipe_sigwake_notify(p) do { } while(0) +#define ipipe_setsched_notify(p) do { } while(0) +#define ipipe_exit_notify(p) do { } while(0) +#define ipipe_init_proc() do { } while(0) +#define ipipe_trap_notify(t,r) 0 +#define ipipe_cleanup_notify(mm) do { } while(0) + +#define spin_lock_hw(lock) spin_lock(lock) +#define spin_unlock_hw(lock) spin_unlock(lock) +#define spin_lock_irq_hw(lock) spin_lock_irq(lock) +#define spin_unlock_irq_hw(lock) spin_unlock_irq(lock) +#define spin_lock_irqsave_hw(lock,flags) spin_lock_irqsave(lock, flags) +#define spin_unlock_irqrestore_hw(lock,flags) spin_unlock_irqrestore(lock, flags) + +#define local_irq_enable_hw_cond() do { } while(0) +#define local_irq_disable_hw_cond() do { } while(0) +#define local_irq_save_hw_cond(flags) do { (void)(flags); } while(0) +#define local_irq_restore_hw_cond(flags) do { } while(0) +#define spin_lock_irqsave_hw_cond(lock,flags) do { (void)(flags); spin_lock(lock); } while(0) +#define spin_unlock_irqrestore_hw_cond(lock,flags) spin_unlock(lock) +#define smp_processor_id_hw() smp_processor_id() + +#define ipipe_irq_lock(irq) do { } while(0) +#define ipipe_irq_unlock(irq) do { } while(0) + +#define ipipe_root_domain_p 1 + +#endif /* CONFIG_IPIPE */ + +#endif /* !__LINUX_IPIPE_H */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/linkage.h linux-2.6.16.5-tcl1-ipipe/include/linux/linkage.h --- linux-2.6.16.5-tcl1/include/linux/linkage.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/linkage.h 2006-07-16 15:01:24.000000000 +0200 @@ -51,4 +51,8 @@ #define fastcall #endif +#ifndef notrace +#define notrace __attribute__((no_instrument_function)) +#endif + #endif diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/preempt.h linux-2.6.16.5-tcl1-ipipe/include/linux/preempt.h --- linux-2.6.16.5-tcl1/include/linux/preempt.h 2006-05-07 15:37:01.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/preempt.h 2006-07-16 15:01:24.000000000 +0200 @@ -27,29 +27,43 @@ asmlinkage void preempt_schedule(void); -#define preempt_disable() \ -do { \ - inc_preempt_count(); \ - barrier(); \ +#ifdef CONFIG_IPIPE +#include <asm/ipipe.h> +extern struct ipipe_domain *ipipe_percpu_domain[], *ipipe_root_domain; +#define ipipe_preempt_guard() (ipipe_percpu_domain[ipipe_processor_id()] == ipipe_root_domain) +#else /* !CONFIG_IPIPE */ +#define ipipe_preempt_guard() 1 +#endif /* CONFIG_IPIPE */ + +#define preempt_disable() \ +do { \ + if (ipipe_preempt_guard()) { \ + inc_preempt_count(); \ + barrier(); \ + } \ } while (0) -#define preempt_enable_no_resched() \ -do { \ - barrier(); \ - dec_preempt_count(); \ +#define preempt_enable_no_resched() \ +do { \ + if (ipipe_preempt_guard()) { \ + barrier(); \ + dec_preempt_count(); \ + } \ } while (0) -#define preempt_check_resched() \ -do { \ - if (unlikely(test_thread_flag(TIF_NEED_RESCHED))) \ - preempt_schedule(); \ +#define preempt_check_resched() \ +do { \ + if (ipipe_preempt_guard()) { \ + if (unlikely(test_thread_flag(TIF_NEED_RESCHED))) \ + preempt_schedule(); \ + } \ } while (0) -#define preempt_enable() \ -do { \ - preempt_enable_no_resched(); \ +#define preempt_enable() \ +do { \ + preempt_enable_no_resched(); \ barrier(); \ - preempt_check_resched(); \ + preempt_check_resched(); \ } while (0) #else diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/sched.h linux-2.6.16.5-tcl1-ipipe/include/linux/sched.h --- linux-2.6.16.5-tcl1/include/linux/sched.h 2006-05-07 16:42:13.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/linux/sched.h 2006-07-16 15:06:26.000000000 +0200 @@ -4,6 +4,7 @@ #include <asm/param.h> /* for HZ */ #include <linux/config.h> +#include <linux/ipipe.h> #include <linux/capability.h> #include <linux/threads.h> #include <linux/kernel.h> @@ -129,6 +130,11 @@ extern unsigned long nr_iowait(void); #define EXIT_DEAD 32 /* in tsk->state again */ #define TASK_NONINTERACTIVE 64 +#ifdef CONFIG_IPIPE +#define TASK_ATOMICSWITCH 512 +#else /* !CONFIG_IPIPE */ +#define TASK_ATOMICSWITCH 0 +#endif /* CONFIG_IPIPE */ #define __set_task_state(tsk, state_value) \ do { (tsk)->state = (state_value); } while (0) @@ -871,6 +877,9 @@ struct task_struct { #endif atomic_t fs_excl; /* holding fs exclusive resources */ struct rcu_head rcu; +#ifdef CONFIG_IPIPE + void *ptd[IPIPE_ROOT_NPTDKEYS]; +#endif }; static inline pid_t process_group(struct task_struct *tsk) diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/init/Kconfig linux-2.6.16.5-tcl1-ipipe/init/Kconfig --- linux-2.6.16.5-tcl1/init/Kconfig 2006-05-07 16:42:14.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/init/Kconfig 2006-07-16 15:01:24.000000000 +0200 @@ -58,6 +58,7 @@ menu "General setup" config LOCALVERSION string "Local version - append to kernel release" + default "-ipipe" help Append an extra string to the end of your kernel version. This will show up when you type uname, for example. diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/init/main.c linux-2.6.16.5-tcl1-ipipe/init/main.c --- linux-2.6.16.5-tcl1/init/main.c 2006-07-15 20:06:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/init/main.c 2006-07-16 15:01:24.000000000 +0200 @@ -488,6 +488,11 @@ asmlinkage void __init start_kernel(void hrtimers_init(); softirq_init(); time_init(); + /* + * We need to wait for the interrupt and time subsystems to be + * initialized before enabling the pipeline. + */ + ipipe_init(); /* * HACK ALERT! This is early. We're enabling the console before @@ -613,6 +618,7 @@ static void __init do_basic_setup(void) #ifdef CONFIG_SYSCTL sysctl_init(); #endif + ipipe_init_proc(); do_initcalls(); } diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/Makefile linux-2.6.16.5-tcl1-ipipe/kernel/Makefile --- linux-2.6.16.5-tcl1/kernel/Makefile 2006-05-07 16:42:15.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/Makefile 2006-07-16 15:01:24.000000000 +0200 @@ -34,6 +34,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_IPIPE) += ipipe/ ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y) # According to Alan Modra <alan@domain.hid>, the -fno-omit-frame-pointer is diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/exit.c linux-2.6.16.5-tcl1-ipipe/kernel/exit.c --- linux-2.6.16.5-tcl1/kernel/exit.c 2006-05-07 16:42:15.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/exit.c 2006-07-16 15:01:24.000000000 +0200 @@ -852,6 +852,7 @@ fastcall NORET_TYPE void do_exit(long co exit_itimers(tsk->signal); acct_process(code); } + ipipe_exit_notify(tsk); exit_mm(tsk); exit_sem(tsk); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/fork.c linux-2.6.16.5-tcl1-ipipe/kernel/fork.c --- linux-2.6.16.5-tcl1/kernel/fork.c 2006-07-13 15:49:59.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/fork.c 2006-07-16 15:15:30.000000000 +0200 @@ -371,6 +371,7 @@ void fastcall __mmdrop(struct mm_struct void mmput(struct mm_struct *mm) { if (atomic_dec_and_test(&mm->mm_users)) { + ipipe_cleanup_notify(mm); exit_aio(mm); exit_mmap(mm); if (!list_empty(&mm->mmlist)) { @@ -1198,6 +1199,14 @@ static task_t *copy_process(unsigned lon spin_unlock(¤t->sighand->siglock); write_unlock_irq(&tasklist_lock); proc_fork_connector(p); +#ifdef CONFIG_IPIPE + { + int k; + + for (k = 0; k < IPIPE_ROOT_NPTDKEYS; k++) + p->ptd[k] = NULL; + } +#endif /* CONFIG_IPIPE */ return p; bad_fork_cleanup_namespace: diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/Kconfig linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Kconfig --- linux-2.6.16.5-tcl1/kernel/ipipe/Kconfig 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Kconfig 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,6 @@ +config IPIPE + bool "Interrupt pipeline" + default y + ---help--- + Activate this option if you want the interrupt pipeline to be + compiled in. diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/Makefile linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Makefile --- linux-2.6.16.5-tcl1/kernel/ipipe/Makefile 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Makefile 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,3 @@ + +obj-$(CONFIG_IPIPE) += core.o generic.o +obj-$(CONFIG_IPIPE_TRACE) += tracer.o diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/core.c linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/core.c --- linux-2.6.16.5-tcl1/kernel/ipipe/core.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/core.c 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,1044 @@ +/* -*- linux-c -*- + * linux/kernel/ipipe/core.c + * + * Copyright (C) 2002-2005 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-independent I-PIPE core support. + */ + +#include <linux/sched.h> +#include <linux/module.h> +#include <linux/kallsyms.h> +#include <linux/interrupt.h> +#ifdef CONFIG_PROC_FS +#include <linux/proc_fs.h> +#endif /* CONFIG_PROC_FS */ + +static struct ipipe_domain ipipe_root = + { .cpudata = {[0 ... IPIPE_NR_CPUS-1] = + { .status = (1<<IPIPE_STALL_FLAG) } } }; + +struct ipipe_domain *ipipe_root_domain = &ipipe_root; + +struct ipipe_domain *ipipe_percpu_domain[IPIPE_NR_CPUS] = + {[0 ... IPIPE_NR_CPUS - 1] = &ipipe_root }; + +ipipe_spinlock_t __ipipe_pipelock = IPIPE_SPIN_LOCK_UNLOCKED; + +LIST_HEAD(__ipipe_pipeline); + +unsigned long __ipipe_virtual_irq_map = 0; + +#ifdef CONFIG_PRINTK +unsigned __ipipe_printk_virq; +#endif /* CONFIG_PRINTK */ + +int __ipipe_event_monitors[IPIPE_NR_EVENTS]; + +/* + * ipipe_init() -- Initialization routine of the IPIPE layer. Called + * by the host kernel early during the boot procedure. + */ +void ipipe_init(void) +{ + struct ipipe_domain *ipd = &ipipe_root; + + __ipipe_check_platform(); /* Do platform dependent checks first. */ + + /* + * A lightweight registration code for the root domain. We are + * running on the boot CPU, hw interrupts are off, and + * secondary CPUs are still lost in space. + */ + + ipd->name = "Linux"; + ipd->domid = IPIPE_ROOT_ID; + ipd->priority = IPIPE_ROOT_PRIO; + + __ipipe_init_stage(ipd); + + INIT_LIST_HEAD(&ipd->p_link); + list_add_tail(&ipd->p_link, &__ipipe_pipeline); + + __ipipe_init_platform(); + +#ifdef CONFIG_PRINTK + __ipipe_printk_virq = ipipe_alloc_virq(); /* Cannot fail here. */ + ipd->irqs[__ipipe_printk_virq].handler = &__ipipe_flush_printk; + ipd->irqs[__ipipe_printk_virq].cookie = NULL; + ipd->irqs[__ipipe_printk_virq].acknowledge = NULL; + ipd->irqs[__ipipe_printk_virq].control = IPIPE_HANDLE_MASK; +#endif /* CONFIG_PRINTK */ + + __ipipe_enable_pipeline(); + + printk(KERN_INFO "I-pipe %s: pipeline enabled.\n", + IPIPE_VERSION_STRING); +} + +void __ipipe_init_stage(struct ipipe_domain *ipd) +{ + int cpuid, n; + + for (cpuid = 0; cpuid < IPIPE_NR_CPUS; cpuid++) { + ipd->cpudata[cpuid].irq_pending_hi = 0; + + for (n = 0; n < IPIPE_IRQ_IWORDS; n++) + ipd->cpudata[cpuid].irq_pending_lo[n] = 0; + + for (n = 0; n < IPIPE_NR_IRQS; n++) { + ipd->cpudata[cpuid].irq_counters[n].total_hits = 0; + ipd->cpudata[cpuid].irq_counters[n].pending_hits = 0; + } + } + + for (n = 0; n < IPIPE_NR_IRQS; n++) { + ipd->irqs[n].acknowledge = NULL; + ipd->irqs[n].handler = NULL; + ipd->irqs[n].control = IPIPE_PASS_MASK; /* Pass but don't handle */ + } + + for (n = 0; n < IPIPE_NR_EVENTS; n++) + ipd->evhand[n] = NULL; + + ipd->evself = 0; + +#ifdef CONFIG_SMP + ipd->irqs[IPIPE_CRITICAL_IPI].acknowledge = &__ipipe_ack_system_irq; + ipd->irqs[IPIPE_CRITICAL_IPI].handler = &__ipipe_do_critical_sync; + ipd->irqs[IPIPE_CRITICAL_IPI].cookie = NULL; + /* Immediately handle in the current domain but *never* pass */ + ipd->irqs[IPIPE_CRITICAL_IPI].control = + IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK|IPIPE_SYSTEM_MASK; +#endif /* CONFIG_SMP */ +} + +void __ipipe_stall_root(void) +{ + ipipe_declare_cpuid; + unsigned long flags; + + ipipe_get_cpu(flags); /* Care for migration. */ + + set_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); + +#ifdef CONFIG_SMP + if (!__ipipe_pipeline_head_p(ipipe_root_domain)) + ipipe_put_cpu(flags); +#else /* CONFIG_SMP */ + if (__ipipe_pipeline_head_p(ipipe_root_domain)) + local_irq_disable_hw(); +#endif /* CONFIG_SMP */ +} + +void __ipipe_cleanup_domain(struct ipipe_domain *ipd) +{ + ipipe_unstall_pipeline_from(ipd); + +#ifdef CONFIG_SMP + { + int cpu; + + for_each_online_cpu(cpu) { + while (ipd->cpudata[cpu].irq_pending_hi != 0) + cpu_relax(); + } + } +#endif /* CONFIG_SMP */ +} + +void __ipipe_unstall_root(void) +{ + ipipe_declare_cpuid; + + local_irq_disable_hw(); + + ipipe_load_cpuid(); + + __clear_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); + + if (ipipe_root_domain->cpudata[cpuid].irq_pending_hi != 0) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + + local_irq_enable_hw(); +} + +unsigned long __ipipe_test_root(void) +{ + unsigned long flags, x; + ipipe_declare_cpuid; + + ipipe_get_cpu(flags); /* Care for migration. */ + x = test_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); + ipipe_put_cpu(flags); + + return x; +} + +unsigned long __ipipe_test_and_stall_root(void) +{ + unsigned long flags, x; + ipipe_declare_cpuid; + + ipipe_get_cpu(flags); /* Care for migration. */ + x = test_and_set_bit(IPIPE_STALL_FLAG, + &ipipe_root_domain->cpudata[cpuid].status); + ipipe_put_cpu(flags); + + return x; +} + +void fastcall __ipipe_restore_root(unsigned long x) +{ + if (x) + __ipipe_stall_root(); + else + __ipipe_unstall_root(); +} + +void fastcall ipipe_stall_pipeline_from(struct ipipe_domain *ipd) +{ + ipipe_declare_cpuid; +#ifdef CONFIG_SMP + unsigned long flags; + + ipipe_lock_cpu(flags); /* Care for migration. */ + + __set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (!__ipipe_pipeline_head_p(ipd)) + ipipe_unlock_cpu(flags); +#else /* CONFIG_SMP */ + set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (__ipipe_pipeline_head_p(ipd)) + local_irq_disable_hw(); +#endif /* CONFIG_SMP */ +} + +unsigned long fastcall ipipe_test_and_stall_pipeline_from(struct ipipe_domain *ipd) +{ + ipipe_declare_cpuid; + unsigned long s; +#ifdef CONFIG_SMP + unsigned long flags; + + ipipe_lock_cpu(flags); /* Care for migration. */ + + s = __test_and_set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (!__ipipe_pipeline_head_p(ipd)) + ipipe_unlock_cpu(flags); +#else /* CONFIG_SMP */ + s = test_and_set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (__ipipe_pipeline_head_p(ipd)) + local_irq_disable_hw(); +#endif /* CONFIG_SMP */ + + return s; +} + +/* + * ipipe_unstall_pipeline_from() -- Unstall the pipeline and + * synchronize pending interrupts for a given domain. See + * __ipipe_walk_pipeline() for more information. + */ +void fastcall ipipe_unstall_pipeline_from(struct ipipe_domain *ipd) +{ + struct list_head *pos; + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + + __clear_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + + if (ipd == ipipe_percpu_domain[cpuid]) + pos = &ipd->p_link; + else + pos = __ipipe_pipeline.next; + + __ipipe_walk_pipeline(pos, cpuid); + + if (__ipipe_pipeline_head_p(ipd)) + local_irq_enable_hw(); + else + ipipe_unlock_cpu(flags); +} + +unsigned long fastcall ipipe_test_and_unstall_pipeline_from(struct ipipe_domain *ipd) +{ + unsigned long flags, x; + ipipe_declare_cpuid; + + ipipe_get_cpu(flags); + x = test_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); + ipipe_unstall_pipeline_from(ipd); + ipipe_put_cpu(flags); + + return x; +} + +void fastcall ipipe_restore_pipeline_from(struct ipipe_domain *ipd, + unsigned long x) +{ + if (x) + ipipe_stall_pipeline_from(ipd); + else + ipipe_unstall_pipeline_from(ipd); +} + +void ipipe_unstall_pipeline_head(void) +{ + struct ipipe_domain *head; + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + head = __ipipe_pipeline_head(); + __clear_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); + + if (unlikely(head->cpudata[cpuid].irq_pending_hi != 0)) { + if (likely(head == ipipe_percpu_domain[cpuid])) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + else + __ipipe_walk_pipeline(&head->p_link, cpuid); + } + + local_irq_enable_hw(); +} + +void fastcall __ipipe_restore_pipeline_head(struct ipipe_domain *head, unsigned long x) +{ + ipipe_declare_cpuid; + unsigned long flags; + + ipipe_lock_cpu(flags); + + if (x) { +#ifdef CONFIG_DEBUG_KERNEL + static int warned; + if (!warned && test_and_set_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status)) { + /* + * Already stalled albeit ipipe_restore_pipeline_head() + * should have detected it? Send a warning once.\n"); + */ + warned = 1; + printk(KERN_WARNING + "I-pipe: ipipe_restore_pipeline_head() optimization failed.\n"); + dump_stack(); + } +#else /* !CONFIG_DEBUG_KERNEL */ + set_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); +#endif /* CONFIG_DEBUG_KERNEL */ + } + else { + __clear_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); + if (unlikely(head->cpudata[cpuid].irq_pending_hi != 0)) { + if (likely(head == ipipe_percpu_domain[cpuid])) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + else + __ipipe_walk_pipeline(&head->p_link, cpuid); + } + local_irq_enable_hw(); + } +} + +/* __ipipe_walk_pipeline(): Plays interrupts pending in the log. Must + be called with local hw interrupts disabled. */ + +void fastcall __ipipe_walk_pipeline(struct list_head *pos, int cpuid) +{ + struct ipipe_domain *this_domain = ipipe_percpu_domain[cpuid]; + + while (pos != &__ipipe_pipeline) { + struct ipipe_domain *next_domain = + list_entry(pos, struct ipipe_domain, p_link); + + if (test_bit + (IPIPE_STALL_FLAG, &next_domain->cpudata[cpuid].status)) + break; /* Stalled stage -- do not go further. */ + + if (next_domain->cpudata[cpuid].irq_pending_hi != 0) { + + if (next_domain == this_domain) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + else { + __ipipe_switch_to(this_domain, next_domain, + cpuid); + + ipipe_load_cpuid(); /* Processor might have changed. */ + + if (this_domain->cpudata[cpuid]. + irq_pending_hi != 0 + && !test_bit(IPIPE_STALL_FLAG, + &this_domain->cpudata[cpuid].status)) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + } + + break; + } else if (next_domain == this_domain) + break; + + pos = next_domain->p_link.next; + } +} + +/* + * ipipe_suspend_domain() -- Suspend the current domain, switching to + * the next one which has pending work down the pipeline. + */ +void ipipe_suspend_domain(void) +{ + struct ipipe_domain *this_domain, *next_domain; + struct list_head *ln; + unsigned long flags; + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + + this_domain = next_domain = ipipe_percpu_domain[cpuid]; + + __clear_bit(IPIPE_STALL_FLAG, &this_domain->cpudata[cpuid].status); + + if (this_domain->cpudata[cpuid].irq_pending_hi != 0) + goto sync_stage; + + for (;;) { + ln = next_domain->p_link.next; + + if (ln == &__ipipe_pipeline) + break; + + next_domain = list_entry(ln, struct ipipe_domain, p_link); + + if (test_bit(IPIPE_STALL_FLAG, + &next_domain->cpudata[cpuid].status)) + break; + + if (next_domain->cpudata[cpuid].irq_pending_hi == 0) + continue; + + ipipe_percpu_domain[cpuid] = next_domain; + +sync_stage: + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + + ipipe_load_cpuid(); /* Processor might have changed. */ + + if (ipipe_percpu_domain[cpuid] != next_domain) + /* + * Something has changed the current domain under our + * feet, recycling the register set; take note. + */ + this_domain = ipipe_percpu_domain[cpuid]; + } + + ipipe_percpu_domain[cpuid] = this_domain; + + ipipe_unlock_cpu(flags); +} + +/* ipipe_alloc_virq() -- Allocate a pipelined virtual/soft interrupt. + * Virtual interrupts are handled in exactly the same way than their + * hw-generated counterparts wrt pipelining. + */ +unsigned ipipe_alloc_virq(void) +{ + unsigned long flags, irq = 0; + int ipos; + + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); + + if (__ipipe_virtual_irq_map != ~0) { + ipos = ffz(__ipipe_virtual_irq_map); + set_bit(ipos, &__ipipe_virtual_irq_map); + irq = ipos + IPIPE_VIRQ_BASE; + } + + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); + + return irq; +} + +/* ipipe_virtualize_irq() -- Attach a handler (and optionally a hw + acknowledge routine) to an interrupt for a given domain. */ + +int ipipe_virtualize_irq(struct ipipe_domain *ipd, + unsigned irq, + ipipe_irq_handler_t handler, + void *cookie, + ipipe_irq_ackfn_t acknowledge, + unsigned modemask) +{ + unsigned long flags; + int err; + + if (irq >= IPIPE_NR_IRQS) + return -EINVAL; + + if (ipd->irqs[irq].control & IPIPE_SYSTEM_MASK) + return -EPERM; + + if (!test_bit(IPIPE_AHEAD_FLAG, &ipd->flags)) + /* Silently unwire interrupts for non-heading domains. */ + modemask &= ~IPIPE_WIRED_MASK; + + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); + + if (handler != NULL) { + + if (handler == IPIPE_SAME_HANDLER) { + handler = ipd->irqs[irq].handler; + cookie = ipd->irqs[irq].cookie; + + if (handler == NULL) { + err = -EINVAL; + goto unlock_and_exit; + } + } else if ((modemask & IPIPE_EXCLUSIVE_MASK) != 0 && + ipd->irqs[irq].handler != NULL) { + err = -EBUSY; + goto unlock_and_exit; + } + + if ((modemask & (IPIPE_SHARED_MASK | IPIPE_PASS_MASK)) == + IPIPE_SHARED_MASK) { + err = -EINVAL; + goto unlock_and_exit; + } + + /* Wired interrupts can only be delivered to domains + * always heading the pipeline. */ + + if ((modemask & IPIPE_WIRED_MASK) != 0) { + if ((modemask & (IPIPE_SHARED_MASK | IPIPE_PASS_MASK | IPIPE_STICKY_MASK)) != 0) { + err = -EINVAL; + goto unlock_and_exit; + } + modemask |= (IPIPE_HANDLE_MASK); + } + + if ((modemask & IPIPE_STICKY_MASK) != 0) + modemask |= IPIPE_HANDLE_MASK; + } else + modemask &= + ~(IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK | + IPIPE_SHARED_MASK | IPIPE_EXCLUSIVE_MASK | IPIPE_WIRED_MASK); + + if (acknowledge == NULL) { + if ((modemask & IPIPE_SHARED_MASK) == 0) { + if (!ipipe_virtual_irq_p(irq)) { + /* Acknowledge handler unspecified for a hw + interrupt -- this is ok in non-shared + management mode, but we will force the use + of the Linux-defined handler instead. */ + acknowledge = ipipe_root_domain->irqs[irq].acknowledge; + } + } + else { + /* A valid acknowledge handler to be called in shared mode + is required when declaring a shared IRQ. */ + err = -EINVAL; + goto unlock_and_exit; + } + } + + ipd->irqs[irq].handler = handler; + ipd->irqs[irq].cookie = cookie; + ipd->irqs[irq].acknowledge = acknowledge; + ipd->irqs[irq].control = modemask; + + if (irq < NR_IRQS && + handler != NULL && + !ipipe_virtual_irq_p(irq) && (modemask & IPIPE_ENABLE_MASK) != 0) { + if (ipd != ipipe_current_domain) { + /* IRQ enable/disable state is domain-sensitive, so we may + not change it for another domain. What is allowed + however is forcing some domain to handle an interrupt + source, by passing the proper 'ipd' descriptor which + thus may be different from ipipe_current_domain. */ + err = -EPERM; + goto unlock_and_exit; + } + + __ipipe_enable_irq(irq); + } + + err = 0; + + unlock_and_exit: + + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); + + return err; +} + +/* ipipe_control_irq() -- Change modes of a pipelined interrupt for + * the current domain. */ + +int ipipe_control_irq(unsigned irq, unsigned clrmask, unsigned setmask) +{ + struct ipipe_domain *ipd; + unsigned long flags; + + if (irq >= IPIPE_NR_IRQS) + return -EINVAL; + + ipd = ipipe_current_domain; + + if (ipd->irqs[irq].control & IPIPE_SYSTEM_MASK) + return -EPERM; + + if (((setmask | clrmask) & IPIPE_SHARED_MASK) != 0) + return -EINVAL; + + if (ipd->irqs[irq].handler == NULL) + setmask &= ~(IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK); + + if ((setmask & IPIPE_STICKY_MASK) != 0) + setmask |= IPIPE_HANDLE_MASK; + + if ((clrmask & (IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK)) != 0) /* If one goes, both go. */ + clrmask |= (IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK); + + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); + + ipd->irqs[irq].control &= ~clrmask; + ipd->irqs[irq].control |= setmask; + + if ((setmask & IPIPE_ENABLE_MASK) != 0) + __ipipe_enable_irq(irq); + else if ((clrmask & IPIPE_ENABLE_MASK) != 0) + __ipipe_disable_irq(irq); + + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); + + return 0; +} + +/* __ipipe_dispatch_event() -- Low-level event dispatcher. */ + +int fastcall __ipipe_dispatch_event (unsigned event, void *data) +{ + struct ipipe_domain *start_domain, *this_domain, *next_domain; + struct list_head *pos, *npos; + unsigned long flags; + ipipe_declare_cpuid; + int propagate = 1; + + ipipe_lock_cpu(flags); + + start_domain = this_domain = ipipe_percpu_domain[cpuid]; + + list_for_each_safe(pos,npos,&__ipipe_pipeline) { + + next_domain = list_entry(pos,struct ipipe_domain,p_link); + + /* + * Note: Domain migration may occur while running + * event or interrupt handlers, in which case the + * current register set is going to be recycled for a + * different domain than the initiating one. We do + * care for that, always tracking the current domain + * descriptor upon return from those handlers. + */ + if (next_domain->evhand[event] != NULL) { + ipipe_percpu_domain[cpuid] = next_domain; + ipipe_unlock_cpu(flags); + propagate = !next_domain->evhand[event](event,start_domain,data); + ipipe_lock_cpu(flags); + if (ipipe_percpu_domain[cpuid] != next_domain) + this_domain = ipipe_percpu_domain[cpuid]; + } + + if (next_domain != ipipe_root_domain && /* NEVER sync the root stage here. */ + next_domain->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG,&next_domain->cpudata[cpuid].status)) { + ipipe_percpu_domain[cpuid] = next_domain; + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + ipipe_load_cpuid(); + if (ipipe_percpu_domain[cpuid] != next_domain) + this_domain = ipipe_percpu_domain[cpuid]; + } + + ipipe_percpu_domain[cpuid] = this_domain; + + if (next_domain == this_domain || !propagate) + break; + } + + ipipe_unlock_cpu(flags); + + return !propagate; +} + +/* + * __ipipe_dispatch_wired -- Wired interrupt dispatcher. Wired + * interrupts are immediately and unconditionally delivered to the + * domain heading the pipeline upon receipt, and such domain must have + * been registered as an invariant head for the system (priority == + * IPIPE_HEAD_PRIORITY). The motivation for using wired interrupts is + * to get an extra-fast dispatching path for those IRQs, by relying on + * a straightforward logic based on assumptions that must always be + * true for invariant head domains. The following assumptions are + * made when dealing with such interrupts: + * + * 1- Wired interrupts are purely dynamic, i.e. the decision to + * propagate them down the pipeline must be done from the head domain + * ISR. + * 2- Wired interrupts cannot be shared or sticky. + * 3- The root domain cannot be an invariant pipeline head, in + * consequence of what the root domain cannot handle wired + * interrupts. + * 4- Wired interrupts must have a valid acknowledge handler for the + * head domain (if needed), and in any case, must not rely on handlers + * provided by lower priority domains during the acknowledge cycle + * (see __ipipe_handle_irq). + * + * Called with hw interrupts off. + */ + +int fastcall __ipipe_dispatch_wired(struct ipipe_domain *head, unsigned irq) +{ + struct ipcpudata *cpudata; + struct ipipe_domain *old; + ipipe_declare_cpuid; + + ipipe_load_cpuid(); + cpudata = &head->cpudata[cpuid]; + cpudata->irq_counters[irq].total_hits++; + + if (test_bit(IPIPE_LOCK_FLAG, &head->irqs[irq].control)) { + /* If we can't process this IRQ right now, we must + * mark it as pending, so that it will get played + * during normal log sync when the corresponding + * interrupt source is eventually unlocked. */ + cpudata->irq_counters[irq].pending_hits++; + return 0; + } + + if (test_bit(IPIPE_STALL_FLAG, &cpudata->status)) { + cpudata->irq_counters[irq].pending_hits++; + __ipipe_set_irq_bit(head, cpuid, irq); + return 0; + } + + old = ipipe_percpu_domain[cpuid]; + ipipe_percpu_domain[cpuid] = head; /* Switch to the head domain. */ + + __set_bit(IPIPE_STALL_FLAG, &cpudata->status); + head->irqs[irq].handler(irq,head->irqs[irq].cookie); /* Call the ISR. */ + __ipipe_run_irqtail(); + __clear_bit(IPIPE_STALL_FLAG, &cpudata->status); + + /* We expect the caller to start a complete pipeline walk upon + * return, so that propagated interrupts will get played. */ + + if (ipipe_percpu_domain[cpuid] == head) + ipipe_percpu_domain[cpuid] = old; /* Back to the preempted domain. */ + + return 1; +} + +/* + * __ipipe_sync_stage() -- Flush the pending IRQs for the current + * domain (and processor). This routine flushes the interrupt log + * (see "Optimistic interrupt protection" from D. Stodolsky et al. for + * more on the deferred interrupt scheme). Every interrupt that + * occurred while the pipeline was stalled gets played. WARNING: + * callers on SMP boxen should always check for CPU migration on + * return of this routine. One can control the kind of interrupts + * which are going to be sync'ed using the syncmask + * parameter. IPIPE_IRQMASK_ANY plays them all, IPIPE_IRQMASK_VIRT + * plays virtual interrupts only. + * + * This routine must be called with hw interrupts off. + */ +void fastcall __ipipe_sync_stage(unsigned long syncmask) +{ + unsigned long mask, submask; + struct ipcpudata *cpudata; + struct ipipe_domain *ipd; + ipipe_declare_cpuid; + int level, rank; + unsigned irq; + + ipipe_load_cpuid(); + ipd = ipipe_percpu_domain[cpuid]; + cpudata = &ipd->cpudata[cpuid]; + + if (__test_and_set_bit(IPIPE_SYNC_FLAG, &cpudata->status)) + return; + + /* + * The policy here is to keep the dispatching code interrupt-free + * by stalling the current stage. If the upper domain handler + * (which we call) wants to re-enable interrupts while in a safe + * portion of the code (e.g. SA_INTERRUPT flag unset for Linux's + * sigaction()), it will have to unstall (then stall again before + * returning to us!) the stage when it sees fit. + */ + while ((mask = (cpudata->irq_pending_hi & syncmask)) != 0) { + level = __ipipe_ffnz(mask); + __clear_bit(level, &cpudata->irq_pending_hi); + + while ((submask = cpudata->irq_pending_lo[level]) != 0) { + rank = __ipipe_ffnz(submask); + irq = (level << IPIPE_IRQ_ISHIFT) + rank; + + if (test_bit(IPIPE_LOCK_FLAG, &ipd->irqs[irq].control)) { + __clear_bit(rank, &cpudata->irq_pending_lo[level]); + continue; + } + + if (--cpudata->irq_counters[irq].pending_hits == 0) + __clear_bit(rank, &cpudata->irq_pending_lo[level]); + + __set_bit(IPIPE_STALL_FLAG, &cpudata->status); + __ipipe_run_isr(ipd, irq, cpuid); +#ifdef CONFIG_SMP + { + int _cpuid = ipipe_processor_id(); + + if (_cpuid != cpuid) { /* Handle CPU migration. */ + /* + * We expect any domain to clear the SYNC bit each + * time it switches in a new task, so that preemptions + * and/or CPU migrations (in the SMP case) over the + * ISR do not lock out the log syncer for some + * indefinite amount of time. In the Linux case, + * schedule() handles this (see kernel/sched.c). For + * this reason, we don't bother clearing it here for + * the source CPU in the migration handling case, + * since it must have scheduled another task in by + * now. + */ + cpuid = _cpuid; + cpudata = &ipd->cpudata[cpuid]; + __set_bit(IPIPE_SYNC_FLAG, &cpudata->status); + } + } +#endif /* CONFIG_SMP */ + + __clear_bit(IPIPE_STALL_FLAG, &cpudata->status); + } + } + + __clear_bit(IPIPE_SYNC_FLAG, &cpudata->status); +} + +#ifdef CONFIG_PROC_FS + +#include <linux/proc_fs.h> + +struct proc_dir_entry *ipipe_proc_root; + +static int __ipipe_version_info_proc(char *page, + char **start, + off_t off, int count, int *eof, void *data) +{ + int len = sprintf(page, "%s\n", IPIPE_VERSION_STRING); + + len -= off; + + if (len <= off + count) + *eof = 1; + + *start = page + off; + + if(len > count) + len = count; + + if(len < 0) + len = 0; + + return len; +} + +static int __ipipe_common_info_proc(char *page, + char **start, + off_t off, int count, int *eof, void *data) +{ + struct ipipe_domain *ipd = (struct ipipe_domain *)data; + unsigned long ctlbits; + unsigned irq, _irq; + char *p = page; + int len; + + spin_lock(&__ipipe_pipelock); + + if (test_bit(IPIPE_AHEAD_FLAG,&ipd->flags)) + p += sprintf(p, "Invariant head"); + else + p += sprintf(p, "Priority=%d", ipd->priority); + + p += sprintf(p, ", Id=0x%.8x\n", ipd->domid); + + irq = 0; + + while (irq < IPIPE_NR_IRQS) { + ctlbits = + (ipd->irqs[irq]. + control & (IPIPE_HANDLE_MASK | IPIPE_PASS_MASK | + IPIPE_STICKY_MASK | IPIPE_WIRED_MASK)); + if (irq >= IPIPE_NR_XIRQS && !ipipe_virtual_irq_p(irq)) { + /* + * There might be a hole between the last external + * IRQ and the first virtual one; skip it. + */ + irq++; + continue; + } + + if (ipipe_virtual_irq_p(irq) + && !test_bit(irq - IPIPE_VIRQ_BASE, + &__ipipe_virtual_irq_map)) { + /* Non-allocated virtual IRQ; skip it. */ + irq++; + continue; + } + + /* + * Attempt to group consecutive IRQ numbers having the + * same virtualization settings in a single line. + */ + + _irq = irq; + + while (++_irq < IPIPE_NR_IRQS) { + if (ipipe_virtual_irq_p(_irq) != + ipipe_virtual_irq_p(irq) + || (ipipe_virtual_irq_p(_irq) + && !test_bit(_irq - IPIPE_VIRQ_BASE, + &__ipipe_virtual_irq_map)) + || ctlbits != (ipd->irqs[_irq]. + control & (IPIPE_HANDLE_MASK | + IPIPE_PASS_MASK | + IPIPE_STICKY_MASK))) + break; + } + + if (_irq == irq + 1) + p += sprintf(p, "irq%u: ", irq); + else + p += sprintf(p, "irq%u-%u: ", irq, _irq - 1); + + /* + * Statuses are as follows: + * o "accepted" means handled _and_ passed down the pipeline. + * o "grabbed" means handled, but the interrupt might be + * terminated _or_ passed down the pipeline depending on + * what the domain handler asks for to the I-pipe. + * o "wired" is basically the same as "grabbed", except that + * the interrupt is unconditionally delivered to an invariant + * pipeline head domain. + * o "passed" means unhandled by the domain but passed + * down the pipeline. + * o "discarded" means unhandled and _not_ passed down the + * pipeline. The interrupt merely disappears from the + * current domain down to the end of the pipeline. + */ + if (ctlbits & IPIPE_HANDLE_MASK) { + if (ctlbits & IPIPE_PASS_MASK) + p += sprintf(p, "accepted"); + else if (ctlbits & IPIPE_WIRED_MASK) + p += sprintf(p, "wired"); + else + p += sprintf(p, "grabbed"); + } else if (ctlbits & IPIPE_PASS_MASK) + p += sprintf(p, "passed"); + else + p += sprintf(p, "discarded"); + + if (ctlbits & IPIPE_STICKY_MASK) + p += sprintf(p, ", sticky"); + + if (ipipe_virtual_irq_p(irq)) + p += sprintf(p, ", virtual"); + + p += sprintf(p, "\n"); + + irq = _irq; + } + + spin_unlock(&__ipipe_pipelock); + + len = p - page; + + if (len <= off + count) + *eof = 1; + + *start = page + off; + + len -= off; + + if (len > count) + len = count; + + if (len < 0) + len = 0; + + return len; +} + +void __ipipe_add_domain_proc(struct ipipe_domain *ipd) +{ + create_proc_read_entry(ipd->name,0444,ipipe_proc_root,&__ipipe_common_info_proc,ipd); +} + +void __ipipe_remove_domain_proc(struct ipipe_domain *ipd) +{ + remove_proc_entry(ipd->name,ipipe_proc_root); +} + +void ipipe_init_proc(void) +{ + ipipe_proc_root = create_proc_entry("ipipe",S_IFDIR, 0); + create_proc_read_entry("version",0444,ipipe_proc_root,&__ipipe_version_info_proc,NULL); + __ipipe_init_trace_proc(); + __ipipe_add_domain_proc(ipipe_root_domain); +} + +#endif /* CONFIG_PROC_FS */ + +EXPORT_SYMBOL(ipipe_virtualize_irq); +EXPORT_SYMBOL(ipipe_control_irq); +EXPORT_SYMBOL(ipipe_suspend_domain); +EXPORT_SYMBOL(ipipe_alloc_virq); +EXPORT_SYMBOL(ipipe_percpu_domain); +EXPORT_SYMBOL(ipipe_root_domain); +EXPORT_SYMBOL(ipipe_stall_pipeline_from); +EXPORT_SYMBOL(ipipe_test_and_stall_pipeline_from); +EXPORT_SYMBOL(ipipe_unstall_pipeline_from); +EXPORT_SYMBOL(ipipe_restore_pipeline_from); +EXPORT_SYMBOL(ipipe_test_and_unstall_pipeline_from); +EXPORT_SYMBOL(ipipe_unstall_pipeline_head); +EXPORT_SYMBOL(__ipipe_restore_pipeline_head); +EXPORT_SYMBOL(__ipipe_unstall_root); +EXPORT_SYMBOL(__ipipe_stall_root); +EXPORT_SYMBOL(__ipipe_restore_root); +EXPORT_SYMBOL(__ipipe_test_and_stall_root); +EXPORT_SYMBOL(__ipipe_test_root); +EXPORT_SYMBOL(__ipipe_dispatch_event); +EXPORT_SYMBOL(__ipipe_dispatch_wired); +EXPORT_SYMBOL(__ipipe_sync_stage); +EXPORT_SYMBOL(__ipipe_pipeline); +EXPORT_SYMBOL(__ipipe_pipelock); +EXPORT_SYMBOL(__ipipe_virtual_irq_map); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/generic.c linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/generic.c --- linux-2.6.16.5-tcl1/kernel/ipipe/generic.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/generic.c 2006-07-16 15:01:24.000000000 +0200 @@ -0,0 +1,396 @@ +/* -*- linux-c -*- + * linux/kernel/ipipe/generic.c + * + * Copyright (C) 2002-2005 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-independent I-PIPE services. + */ + +#include <linux/version.h> +#include <linux/module.h> +#include <linux/init.h> +#include <linux/kernel.h> +#include <linux/sched.h> +#ifdef CONFIG_PROC_FS +#include <linux/proc_fs.h> +#endif /* CONFIG_PROC_FS */ + +MODULE_DESCRIPTION("I-pipe"); +MODULE_LICENSE("GPL"); + +static int __ipipe_ptd_key_count; + +static unsigned long __ipipe_ptd_key_map; + +/* ipipe_register_domain() -- Link a new domain to the pipeline. */ + +int ipipe_register_domain(struct ipipe_domain *ipd, + struct ipipe_domain_attr *attr) +{ + struct list_head *pos; + unsigned long flags; + + if (ipipe_current_domain != ipipe_root_domain) { + printk(KERN_WARNING + "I-pipe: Only the root domain may register a new domain.\n"); + return -EPERM; + } + + if (attr->priority == IPIPE_HEAD_PRIORITY && + test_bit(IPIPE_AHEAD_FLAG,&__ipipe_pipeline_head()->flags)) + return -EAGAIN; /* Cannot override current head. */ + + flags = ipipe_critical_enter(NULL); + + list_for_each(pos, &__ipipe_pipeline) { + struct ipipe_domain *_ipd = + list_entry(pos, struct ipipe_domain, p_link); + if (_ipd->domid == attr->domid) + break; + } + + ipipe_critical_exit(flags); + + if (pos != &__ipipe_pipeline) + /* A domain with the given id already exists -- fail. */ + return -EBUSY; + + ipd->name = attr->name; + ipd->domid = attr->domid; + ipd->pdd = attr->pdd; + ipd->flags = 0; + + if (attr->priority == IPIPE_HEAD_PRIORITY) { + ipd->priority = INT_MAX; + __set_bit(IPIPE_AHEAD_FLAG,&ipd->flags); + } + else + ipd->priority = attr->priority; + + __ipipe_init_stage(ipd); + + INIT_LIST_HEAD(&ipd->p_link); + +#ifdef CONFIG_PROC_FS + __ipipe_add_domain_proc(ipd); +#endif /* CONFIG_PROC_FS */ + + flags = ipipe_critical_enter(NULL); + + list_for_each(pos, &__ipipe_pipeline) { + struct ipipe_domain *_ipd = + list_entry(pos, struct ipipe_domain, p_link); + if (ipd->priority > _ipd->priority) + break; + } + + list_add_tail(&ipd->p_link, pos); + + ipipe_critical_exit(flags); + + printk(KERN_WARNING "I-pipe: Domain %s registered.\n", ipd->name); + + /* + * Finally, allow the new domain to perform its initialization + * chores. + */ + + if (attr->entry != NULL) { + ipipe_declare_cpuid; + + ipipe_lock_cpu(flags); + + ipipe_percpu_domain[cpuid] = ipd; + attr->entry(); + ipipe_percpu_domain[cpuid] = ipipe_root_domain; + + ipipe_load_cpuid(); /* Processor might have changed. */ + + if (ipipe_root_domain->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG, + &ipipe_root_domain->cpudata[cpuid].status)) + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); + + ipipe_unlock_cpu(flags); + } + + return 0; +} + +/* ipipe_unregister_domain() -- Remove a domain from the pipeline. */ + +int ipipe_unregister_domain(struct ipipe_domain *ipd) +{ + unsigned long flags; + + if (ipipe_current_domain != ipipe_root_domain) { + printk(KERN_WARNING + "I-pipe: Only the root domain may unregister a domain.\n"); + return -EPERM; + } + + if (ipd == ipipe_root_domain) { + printk(KERN_WARNING + "I-pipe: Cannot unregister the root domain.\n"); + return -EPERM; + } +#ifdef CONFIG_SMP + { + int nr_cpus = num_online_cpus(), _cpuid; + unsigned irq; + + /* + * In the SMP case, wait for the logged events to drain on + * other processors before eventually removing the domain + * from the pipeline. + */ + + ipipe_unstall_pipeline_from(ipd); + + flags = ipipe_critical_enter(NULL); + + for (irq = 0; irq < IPIPE_NR_IRQS; irq++) { + clear_bit(IPIPE_HANDLE_FLAG, &ipd->irqs[irq].control); + clear_bit(IPIPE_STICKY_FLAG, &ipd->irqs[irq].control); + set_bit(IPIPE_PASS_FLAG, &ipd->irqs[irq].control); + } + + ipipe_critical_exit(flags); + + for (_cpuid = 0; _cpuid < nr_cpus; _cpuid++) + for (irq = 0; irq < IPIPE_NR_IRQS; irq++) + while (ipd->cpudata[_cpuid].irq_counters[irq].pending_hits > 0) + cpu_relax(); + } +#endif /* CONFIG_SMP */ + +#ifdef CONFIG_PROC_FS + __ipipe_remove_domain_proc(ipd); +#endif /* CONFIG_PROC_FS */ + + /* + * Simply remove the domain from the pipeline and we are almost done. + */ + + flags = ipipe_critical_enter(NULL); + list_del_init(&ipd->p_link); + ipipe_critical_exit(flags); + + __ipipe_cleanup_domain(ipd); + + printk(KERN_WARNING "I-pipe: Domain %s unregistered.\n", ipd->name); + + return 0; +} + +/* + * ipipe_propagate_irq() -- Force a given IRQ propagation on behalf of + * a running interrupt handler to the next domain down the pipeline. + * ipipe_schedule_irq() -- Does almost the same as above, but attempts + * to pend the interrupt for the current domain first. + */ +int fastcall __ipipe_schedule_irq(unsigned irq, struct list_head *head) +{ + struct list_head *ln; + unsigned long flags; + ipipe_declare_cpuid; + + if (irq >= IPIPE_NR_IRQS || + (ipipe_virtual_irq_p(irq) + && !test_bit(irq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map))) + return -EINVAL; + + ipipe_lock_cpu(flags); + + ln = head; + + while (ln != &__ipipe_pipeline) { + struct ipipe_domain *ipd = + list_entry(ln, struct ipipe_domain, p_link); + + if (test_bit(IPIPE_HANDLE_FLAG, &ipd->irqs[irq].control)) { + ipd->cpudata[cpuid].irq_counters[irq].total_hits++; + ipd->cpudata[cpuid].irq_counters[irq].pending_hits++; + __ipipe_set_irq_bit(ipd, cpuid, irq); + ipipe_unlock_cpu(flags); + return 1; + } + + ln = ipd->p_link.next; + } + + ipipe_unlock_cpu(flags); + + return 0; +} + +/* ipipe_free_virq() -- Release a virtual/soft interrupt. */ + +int ipipe_free_virq(unsigned virq) +{ + if (!ipipe_virtual_irq_p(virq)) + return -EINVAL; + + clear_bit(virq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map); + + return 0; +} + +void ipipe_init_attr(struct ipipe_domain_attr *attr) +{ + attr->name = "anon"; + attr->domid = 1; + attr->entry = NULL; + attr->priority = IPIPE_ROOT_PRIO; + attr->pdd = NULL; +} + +/* + * ipipe_catch_event() -- Interpose or remove an event handler for a + * given domain. + */ +ipipe_event_handler_t ipipe_catch_event(struct ipipe_domain *ipd, + unsigned event, + ipipe_event_handler_t handler) +{ + ipipe_event_handler_t old_handler; + int self = 0; + + if (event & IPIPE_EVENT_SELF) { + event &= ~IPIPE_EVENT_SELF; + self = 1; + } + + if (event >= IPIPE_NR_EVENTS) + return NULL; + + if (!(old_handler = xchg(&ipd->evhand[event],handler))) { + if (handler) { + if (self) + ipd->evself |= (1LL << event); + else + __ipipe_event_monitors[event]++; + } + } + else if (!handler) { + if (ipd->evself & (1LL << event)) + ipd->evself &= ~(1LL << event); + else + __ipipe_event_monitors[event]--; + } else if ((ipd->evself & (1LL << event)) && !self) { + __ipipe_event_monitors[event]++; + ipd->evself &= ~(1LL << event); + } else if (!(ipd->evself & (1LL << event)) && self) { + __ipipe_event_monitors[event]--; + ipd->evself |= (1LL << event); + } + + return old_handler; +} + +cpumask_t ipipe_set_irq_affinity (unsigned irq, cpumask_t cpumask) +{ +#ifdef CONFIG_SMP + if (irq >= IPIPE_NR_XIRQS) + /* Allow changing affinity of external IRQs only. */ + return CPU_MASK_NONE; + + if (num_online_cpus() > 1) + return __ipipe_set_irq_affinity(irq,cpumask); +#endif /* CONFIG_SMP */ + + return CPU_MASK_NONE; +} + +int fastcall ipipe_send_ipi (unsigned ipi, cpumask_t cpumask) + +{ +#ifdef CONFIG_SMP + return __ipipe_send_ipi(ipi,cpumask); +#else /* !CONFIG_SMP */ + return -EINVAL; +#endif /* CONFIG_SMP */ +} + +int ipipe_alloc_ptdkey (void) +{ + unsigned long flags; + int key = -1; + + spin_lock_irqsave_hw(&__ipipe_pipelock,flags); + + if (__ipipe_ptd_key_count < IPIPE_ROOT_NPTDKEYS) { + key = ffz(__ipipe_ptd_key_map); + set_bit(key,&__ipipe_ptd_key_map); + __ipipe_ptd_key_count++; + } + + spin_unlock_irqrestore_hw(&__ipipe_pipelock,flags); + + return key; +} + +int ipipe_free_ptdkey (int key) +{ + unsigned long flags; + + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) + return -EINVAL; + + spin_lock_irqsave_hw(&__ipipe_pipelock,flags); + + if (test_and_clear_bit(key,&__ipipe_ptd_key_map)) + __ipipe_ptd_key_count--; + + spin_unlock_irqrestore_hw(&__ipipe_pipelock,flags); + + return 0; +} + +int fastcall ipipe_set_ptd (int key, void *value) + +{ + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) + return -EINVAL; + + current->ptd[key] = value; + + return 0; +} + +void fastcall *ipipe_get_ptd (int key) + +{ + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) + return NULL; + + return current->ptd[key]; +} + +EXPORT_SYMBOL(ipipe_register_domain); +EXPORT_SYMBOL(ipipe_unregister_domain); +EXPORT_SYMBOL(ipipe_free_virq); +EXPORT_SYMBOL(ipipe_init_attr); +EXPORT_SYMBOL(ipipe_catch_event); +EXPORT_SYMBOL(ipipe_alloc_ptdkey); +EXPORT_SYMBOL(ipipe_free_ptdkey); +EXPORT_SYMBOL(ipipe_set_ptd); +EXPORT_SYMBOL(ipipe_get_ptd); +EXPORT_SYMBOL(ipipe_set_irq_affinity); +EXPORT_SYMBOL(ipipe_send_ipi); +EXPORT_SYMBOL(__ipipe_schedule_irq); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/irq/handle.c linux-2.6.16.5-tcl1-ipipe/kernel/irq/handle.c --- linux-2.6.16.5-tcl1/kernel/irq/handle.c 2006-05-07 15:37:02.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/irq/handle.c 2006-07-16 15:01:24.000000000 +0200 @@ -81,6 +81,17 @@ fastcall int handle_IRQ_event(unsigned i { int ret, retval = 0, status = 0; +#ifdef CONFIG_IPIPE + /* + * If processing a timer tick, pass the original regs as + * collected during preemption and not our phony - always + * kernel-originated - frame, so that we don't wreck the + * profiling code. + */ + if (__ipipe_tick_irq == irq) + regs = __ipipe_tick_regs + smp_processor_id(); +#endif /* CONFIG_IPIPE */ + if (!(action->flags & SA_INTERRUPT)) local_irq_enable(); @@ -117,16 +128,20 @@ fastcall unsigned int __do_IRQ(unsigned /* * No locking required for CPU-local interrupts: */ +#ifndef CONFIG_IPIPE if (desc->handler->ack) desc->handler->ack(irq); +#endif /* !CONFIG_IPIPE */ action_ret = handle_IRQ_event(irq, regs, desc->action); desc->handler->end(irq); return 1; } spin_lock(&desc->lock); +#ifndef CONFIG_IPIPE if (desc->handler->ack) desc->handler->ack(irq); +#endif /* !CONFIG_IPIPE */ /* * REPLAY is when Linux resends an IRQ that was dropped earlier * WAITING is used by probe to mark irqs that are being tested diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/printk.c linux-2.6.16.5-tcl1-ipipe/kernel/printk.c --- linux-2.6.16.5-tcl1/kernel/printk.c 2006-07-15 20:06:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/printk.c 2006-07-16 15:01:24.000000000 +0200 @@ -520,6 +520,78 @@ __attribute__((weak)) unsigned long long * printf(3) */ +#ifdef CONFIG_IPIPE + +static ipipe_spinlock_t __ipipe_printk_lock = IPIPE_SPIN_LOCK_UNLOCKED; + +static int __ipipe_printk_fill; + +static char __ipipe_printk_buf[__LOG_BUF_LEN]; + +void __ipipe_flush_printk (unsigned virq, void *cookie) +{ + char *p = __ipipe_printk_buf; + int len, lmax, out = 0; + unsigned long flags; + + goto start; + + do { + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); + start: + lmax = __ipipe_printk_fill; + while (out < lmax) { + len = strlen(p) + 1; + printk("%s",p); + p += len; + out += len; + } + spin_lock_irqsave_hw(&__ipipe_printk_lock,flags); + } + while (__ipipe_printk_fill != lmax); + + __ipipe_printk_fill = 0; + + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); +} + +asmlinkage int printk(const char *fmt, ...) +{ + int r, fbytes, oldcount; + unsigned long flags; + va_list args; + + va_start(args, fmt); + + if (ipipe_current_domain == ipipe_root_domain || + test_bit(IPIPE_SPRINTK_FLAG,&ipipe_current_domain->flags) || + oops_in_progress) { + r = vprintk(fmt, args); + goto out; + } + + spin_lock_irqsave_hw(&__ipipe_printk_lock,flags); + + oldcount = __ipipe_printk_fill; + fbytes = __LOG_BUF_LEN - oldcount; + + if (fbytes > 1) { + r = vscnprintf(__ipipe_printk_buf + __ipipe_printk_fill, + fbytes, fmt, args) + 1; /* account for the null byte */ + __ipipe_printk_fill += r; + } else + r = 0; + + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); + + if (oldcount == 0) + ipipe_trigger_irq(__ipipe_printk_virq); +out: + va_end(args); + + return r; +} +#else /* !CONFIG_IPIPE */ asmlinkage int printk(const char *fmt, ...) { va_list args; @@ -531,6 +603,7 @@ asmlinkage int printk(const char *fmt, . return r; } +#endif /* CONFIG_IPIPE */ /* cpu currently holding logbuf_lock */ static volatile unsigned int printk_cpu = UINT_MAX; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/sched.c linux-2.6.16.5-tcl1-ipipe/kernel/sched.c --- linux-2.6.16.5-tcl1/kernel/sched.c 2006-07-13 15:49:59.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/sched.c 2006-07-16 15:01:24.000000000 +0200 @@ -2872,13 +2872,18 @@ asmlinkage void __sched schedule(void) unsigned long run_time; int cpu, idx, new_prio; +#ifdef CONFIG_IPIPE + if (unlikely(ipipe_current_domain != ipipe_root_domain)) { + goto need_resched; + } +#endif /* CONFIG_IPIPE */ /* * Test if we are atomic. Since do_exit() needs to call into * schedule() atomically, we ignore that path for now. * Otherwise, whine if we are scheduling when we should not be. */ if (likely(!current->exit_state)) { - if (unlikely(in_atomic())) { + if (unlikely(!(current->state & TASK_ATOMICSWITCH) && in_atomic())) { printk(KERN_ERR "scheduling while atomic: " "%s/0x%08x/%d\n", current->comm, preempt_count(), current->pid); @@ -2887,8 +2892,19 @@ asmlinkage void __sched schedule(void) } profile_hit(SCHED_PROFILING, __builtin_return_address(0)); + if (unlikely(current->state & TASK_ATOMICSWITCH)) { + current->state &= ~TASK_ATOMICSWITCH; + goto preemption_off; + } need_resched: preempt_disable(); +preemption_off: +#ifdef CONFIG_IPIPE + if (unlikely(ipipe_current_domain != ipipe_root_domain)) { + preempt_enable(); + return; + } +#endif /* CONFIG_IPIPE */ prev = current; release_kernel_lock(prev); need_resched_nonpreemptible: @@ -3027,6 +3043,8 @@ switch_tasks: prepare_task_switch(rq, next); prev = context_switch(rq, prev, next); barrier(); + if (task_hijacked(prev)) + return; /* * this_rq must be evaluated again because prev may have moved * CPUs since it called schedule(), thus the 'rq' on its stack @@ -3059,6 +3077,11 @@ asmlinkage void __sched preempt_schedule struct task_struct *task = current; int saved_lock_depth; #endif +#ifdef CONFIG_IPIPE + /* Do not reschedule over non-Linux domains. */ + if (ipipe_current_domain != ipipe_root_domain) + return; +#endif /* CONFIG_IPIPE */ /* * If there is a non-zero preempt_count or interrupts are disabled, * we do not want to preempt the current task. Just return.. @@ -3697,6 +3720,7 @@ recheck: deactivate_task(p, rq); oldprio = p->prio; __setscheduler(p, policy, param->sched_priority); + ipipe_setsched_notify(p); if (array) { __activate_task(p, rq); /* @@ -6165,3 +6189,50 @@ void set_curr_task(int cpu, task_t *p) } #endif + +#ifdef CONFIG_IPIPE + +int ipipe_setscheduler_root (struct task_struct *p, int policy, int prio) +{ + prio_array_t *array; + unsigned long flags; + runqueue_t *rq; + int oldprio; + + rq = task_rq_lock(p, &flags); + array = p->array; + if (array) + deactivate_task(p, rq); + oldprio = p->prio; + __setscheduler(p, policy, prio); + if (array) { + __activate_task(p, rq); + if (task_running(rq, p)) { + if (p->prio > oldprio) + resched_task(rq->curr); + } else if (TASK_PREEMPTS_CURR(p, rq)) + resched_task(rq->curr); + } + task_rq_unlock(rq, &flags); + + return 0; +} + +EXPORT_SYMBOL(ipipe_setscheduler_root); + +int ipipe_reenter_root (struct task_struct *prev, int policy, int prio) +{ + finish_task_switch(this_rq(), prev); + if (reacquire_kernel_lock(current) < 0) + ; + preempt_enable_no_resched(); + + if (current->policy != policy || current->rt_priority != prio) + return ipipe_setscheduler_root(current,policy,prio); + + return 0; +} + +EXPORT_SYMBOL(ipipe_reenter_root); + +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/signal.c linux-2.6.16.5-tcl1-ipipe/kernel/signal.c --- linux-2.6.16.5-tcl1/kernel/signal.c 2006-07-13 15:49:59.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/kernel/signal.c 2006-07-16 15:01:24.000000000 +0200 @@ -604,6 +604,7 @@ void signal_wake_up(struct task_struct * unsigned int mask; set_tsk_thread_flag(t, TIF_SIGPENDING); + ipipe_sigwake_notify(t); /* TIF_SIGPENDING must be set first. */ /* * For SIGKILL, we want to wake it up in the stopped/traced case. diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/lib/smp_processor_id.c linux-2.6.16.5-tcl1-ipipe/lib/smp_processor_id.c --- linux-2.6.16.5-tcl1/lib/smp_processor_id.c 2006-05-07 15:37:02.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/lib/smp_processor_id.c 2006-07-16 15:01:24.000000000 +0200 @@ -13,6 +13,11 @@ unsigned int debug_smp_processor_id(void int this_cpu = raw_smp_processor_id(); cpumask_t this_mask; +#ifdef CONFIG_IPIPE + if (ipipe_current_domain != ipipe_root_domain) + return this_cpu; +#endif /* CONFIG_IPIPE */ + if (likely(preempt_count)) goto out; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/lib/spinlock_debug.c linux-2.6.16.5-tcl1-ipipe/lib/spinlock_debug.c --- linux-2.6.16.5-tcl1/lib/spinlock_debug.c 2006-05-07 16:42:15.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/lib/spinlock_debug.c 2006-07-16 15:01:24.000000000 +0200 @@ -10,6 +10,7 @@ #include <linux/spinlock.h> #include <linux/interrupt.h> #include <linux/delay.h> +#include <linux/module.h> static void spin_bug(spinlock_t *lock, const char *msg) { @@ -96,6 +97,8 @@ void _raw_spin_lock(spinlock_t *lock) debug_spin_lock_after(lock); } +EXPORT_SYMBOL(_raw_spin_lock); + int _raw_spin_trylock(spinlock_t *lock) { int ret = __raw_spin_trylock(&lock->raw_lock); @@ -111,12 +114,16 @@ int _raw_spin_trylock(spinlock_t *lock) return ret; } +EXPORT_SYMBOL(_raw_spin_trylock); + void _raw_spin_unlock(spinlock_t *lock) { debug_spin_unlock(lock); __raw_spin_unlock(&lock->raw_lock); } +EXPORT_SYMBOL(_raw_spin_unlock); + static void rwlock_bug(rwlock_t *lock, const char *msg) { static long print_once = 1; @@ -167,6 +174,8 @@ void _raw_read_lock(rwlock_t *lock) __read_lock_debug(lock); } +EXPORT_SYMBOL(_raw_read_lock); + int _raw_read_trylock(rwlock_t *lock) { int ret = __raw_read_trylock(&lock->raw_lock); @@ -180,12 +189,16 @@ int _raw_read_trylock(rwlock_t *lock) return ret; } +EXPORT_SYMBOL(_raw_read_trylock); + void _raw_read_unlock(rwlock_t *lock) { RWLOCK_BUG_ON(lock->magic != RWLOCK_MAGIC, lock, "bad magic"); __raw_read_unlock(&lock->raw_lock); } +EXPORT_SYMBOL(_raw_read_unlock); + static inline void debug_write_lock_before(rwlock_t *lock) { RWLOCK_BUG_ON(lock->magic != RWLOCK_MAGIC, lock, "bad magic"); @@ -241,6 +254,8 @@ void _raw_write_lock(rwlock_t *lock) debug_write_lock_after(lock); } +EXPORT_SYMBOL(_raw_write_lock); + int _raw_write_trylock(rwlock_t *lock) { int ret = __raw_write_trylock(&lock->raw_lock); @@ -256,8 +271,12 @@ int _raw_write_trylock(rwlock_t *lock) return ret; } +EXPORT_SYMBOL(_raw_write_trylock); + void _raw_write_unlock(rwlock_t *lock) { debug_write_unlock(lock); __raw_write_unlock(&lock->raw_lock); } + +EXPORT_SYMBOL(_raw_write_unlock); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/mm/vmalloc.c linux-2.6.16.5-tcl1-ipipe/mm/vmalloc.c --- linux-2.6.16.5-tcl1/mm/vmalloc.c 2006-05-07 15:37:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/mm/vmalloc.c 2006-07-16 15:01:24.000000000 +0200 @@ -19,6 +19,7 @@ #include <asm/uaccess.h> #include <asm/tlbflush.h> +#include <asm/pgalloc.h> DEFINE_RWLOCK(vmlist_lock); @@ -148,10 +149,14 @@ int map_vm_area(struct vm_struct *area, BUG_ON(addr >= end); pgd = pgd_offset_k(addr); do { + pgd_t oldpgd; + memcpy(&oldpgd,pgd,sizeof(pgd_t)); next = pgd_addr_end(addr, end); err = vmap_pud_range(pgd, addr, next, prot, pages); if (err) break; + if (pgd_val(oldpgd) != pgd_val(*pgd)) + set_pgdir(addr, *pgd); } while (pgd++, addr = next, addr != end); flush_cache_vmap((unsigned long) area->addr, end); return err; ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-17 16:56 ` Gilles Chanteperdrix @ 2006-07-31 8:34 ` Bart Jonkers 2006-07-31 9:20 ` Detlef Vollmann 0 siblings, 1 reply; 36+ messages in thread From: Bart Jonkers @ 2006-07-31 8:34 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai On Mon, 2006-07-17 at 18:56 +0200, Gilles Chanteperdrix wrote: > Philippe Gerum wrote: > > On Mon, 2006-07-17 at 18:17 +0200, Gilles Chanteperdrix wrote: > > > replaced PSR_I_BIT with its value. I now wonder if it is the proper fix, > > > or if __xchg should be fixed to use local_irq_save_hw and > > > local_irq_restore_hw instead, as is the case for the atomic operations > > > defined in atomic.h and bitops.h. > > > > Definitely, it should be fixed to use hw masking ops. > > Attached a corrected patch for 2.6.16. Hey, I tried this patch together with your ipipe-sa1100-pxa patch on my pxa-255 based platform. There are two adaptions that I need to make in arch/arm/mach-pxa/time.c to compile the kernel properly: - include linux/module.h to remove warnings about EXPORT_SYMBOL - replace sa1100_timer_initialized with pxa_timer_initialized When I boot the kernel with ipipe enabled, linux receives no interrupts anymore. Any idea to solve this? thanks in advance, Bart > > plain text document attachment > (adeos-ipipe-2.6.16-arm-unofficial2.patch) > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/Kconfig linux-2.6.16.5-tcl1-ipipe/arch/arm/Kconfig > --- linux-2.6.16.5-tcl1/arch/arm/Kconfig 2006-07-15 20:06:02.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/Kconfig 2006-07-16 15:01:24.000000000 +0200 > @@ -401,6 +401,8 @@ config LOCAL_TIMERS > accounting to be spread across the timer interval, preventing a > "thundering herd" at every timer tick. > > +source "kernel/ipipe/Kconfig" > + > config PREEMPT > bool "Preemptible Kernel (EXPERIMENTAL)" > depends on EXPERIMENTAL > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/Makefile linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/Makefile > --- linux-2.6.16.5-tcl1/arch/arm/kernel/Makefile 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/Makefile 2006-07-16 15:19:02.000000000 +0200 > @@ -21,6 +21,7 @@ obj-$(CONFIG_ISA_DMA) += dma-isa.o > obj-$(CONFIG_PCI) += bios32.o > obj-$(CONFIG_SMP) += smp.o > obj-$(CONFIG_OABI_COMPAT) += sys_oabi-compat.o > +obj-$(CONFIG_IPIPE) += ipipe-core.o ipipe-root.o > > obj-$(CONFIG_IWMMXT) += iwmmxt.o > AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-armv.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-armv.S > --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-armv.S 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-armv.S 2006-07-16 15:19:54.000000000 +0200 > @@ -4,6 +4,7 @@ > * Copyright (C) 1996,1997,1998 Russell King. > * ARM700 fix by Matthew Godbolt (linux-user@domain.hid) > * nommu support by Hyok S. Choi (hyok.choi@domain.hid) > + * Copyright (C) 2005 Stelian Pop. > * > * This program is free software; you can redistribute it and/or modify > * it under the terms of the GNU General Public License version 2 as > @@ -33,7 +34,11 @@ > @ routine called with r0 = irq number, r1 = struct pt_regs * > @ > adrne lr, 1b > +#ifdef CONFIG_IPIPE > + bne __ipipe_grab_irq > +#else > bne asm_do_IRQ > +#endif > > #ifdef CONFIG_SMP > /* > @@ -199,6 +204,11 @@ __irq_svc: > #endif > > irq_handler > +#ifdef CONFIG_IPIPE > + cmp r0, #0 > + beq __ipipe_fast_svc_irq_exit > +#endif > + > #ifdef CONFIG_PREEMPT > ldr r0, [tsk, #TI_FLAGS] @ get flags > tst r0, #_TIF_NEED_RESCHED > @@ -209,6 +219,9 @@ preempt_return: > teq r0, r7 > strne r0, [r0, -r0] @ bug() > #endif > +#ifdef CONFIG_IPIPE > +__ipipe_fast_svc_irq_exit: > +#endif > ldr r0, [sp, #S_PSR] @ irqs are already disabled > msr spsr_cxsf, r0 > ldmia sp, {r0 - pc}^ @ load r0 - pc, cpsr > @@ -237,6 +250,12 @@ svc_preempt: > __und_svc: > svc_entry > > +#ifdef CONFIG_IPIPE > + mov r1, sp @ r0 = trapno, r1 = ®s > + bl __ipipe_dispatch_event @ branch to trap handler > + cmp r0, #0 > + bne 1f > +#endif /* CONFIG_IPIPE */ > @ > @ call emulation code, which returns using r9 if it has emulated > @ the instruction, or the more conventional lr if we are to treat > @@ -406,6 +425,12 @@ __irq_usr: > #endif > > irq_handler > +#ifdef CONFIG_IPIPE > + cmp r0, #0 > + bne __ipipe_usr_irq_continue > + slow_restore_user_regs @ Fast exit path over non-root domains > +__ipipe_usr_irq_continue: > +#endif > #ifdef CONFIG_PREEMPT > ldr r0, [tsk, #TI_PREEMPT] > str r8, [tsk, #TI_PREEMPT] > @@ -499,8 +524,8 @@ call_fpe: > mov pc, lr @ CP#8 > mov pc, lr @ CP#9 > #ifdef CONFIG_VFP > - b do_vfp @ CP#10 (VFP) > - b do_vfp @ CP#11 (VFP) > + b _do_vfp @ CP#10 (VFP) > + b _do_vfp @ CP#11 (VFP) > #else > mov pc, lr @ CP#10 (VFP) > mov pc, lr @ CP#11 (VFP) > @@ -511,10 +536,34 @@ call_fpe: > mov pc, lr @ CP#15 (Control) > > do_fpe: > +#ifdef CONFIG_IPIPE > + mov r4, r0 > + mov r0, #5 @ == IPIPE_TRAP_FPU > + mov r1, sp @ r0 = trapno, r1 = ®s > + bl __ipipe_dispatch_event @ branch to trap handler > + cmp r0, #0 > + ldrne pc, [r9] > + mov r0, r4 > +#endif > ldr r4, .LCfp > add r10, r10, #TI_FPSTATE @ r10 = workspace > ldr pc, [r4] @ Call FP module USR entry point > > +#ifdef CONFIG_VFP > +_do_vfp: > +#ifdef CONFIG_IPIPE > + mov r4, r0 > + mov r0, #6 @ == IPIPE_TRAP_VFP > + mov r1, sp @ r0 = trapno, r1 = ®s > + bl __ipipe_dispatch_event @ branch to trap handler > + cmp r0, #0 > + ldrne pc, [r9] > + mov r0, r4 > +#endif > + b do_vfp > +#endif > + > + > /* > * The FP module is called with these registers set: > * r0 = instruction > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-common.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-common.S > --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-common.S 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-common.S 2006-07-16 16:15:08.000000000 +0200 > @@ -2,6 +2,7 @@ > * linux/arch/arm/kernel/entry-common.S > * > * Copyright (C) 2000 Russell King > + * Copyright (C) 2005 Stelian Pop. > * > * This program is free software; you can redistribute it and/or modify > * it under the terms of the GNU General Public License version 2 as > @@ -20,13 +21,18 @@ > * possible here, and this includes saving r0 back into the SVC > * stack. > */ > +#ifdef CONFIG_IPIPE > +__ipipe_ret_fast_syscall: > + ldr r0, [sp, #S_R0+S_OFF] @ returned r0 > + /* fall through */ > +#endif > ret_fast_syscall: > disable_irq @ disable interrupts > ldr r1, [tsk, #TI_FLAGS] > tst r1, #_TIF_WORK_MASK > bne fast_work_pending > > - @ fast_restore_user_regs > +fast_restore_user_regs: > ldr r1, [sp, #S_OFF + S_PSR] @ get calling cpsr > ldr lr, [sp, #S_OFF + S_PC]! @ get pc > msr spsr_cxsf, r1 @ save in spsr_svc > @@ -35,6 +41,13 @@ ret_fast_syscall: > add sp, sp, #S_FRAME_SIZE - S_PC > movs pc, lr @ return & move spsr_svc into cpsr > > +#ifdef CONFIG_IPIPE > +__ipipe_fast_exit_syscall: > + ldr r0, [sp, #S_R0+S_OFF] @ returned r0 > + disable_irq @ disable interrupts > + b fast_restore_user_regs > +#endif /* CONFIG_IPIPE */ > + > /* > * Ok, we need to do extra processing, enter the slow path. > */ > @@ -62,19 +75,15 @@ ret_slow_syscall: > tst r1, #_TIF_WORK_MASK > bne work_pending > no_work_pending: > - @ slow_restore_user_regs > - ldr r1, [sp, #S_PSR] @ get calling cpsr > - ldr lr, [sp, #S_PC]! @ get pc > - msr spsr_cxsf, r1 @ save in spsr_svc > - ldmdb sp, {r0 - lr}^ @ get calling r1 - lr > - mov r0, r0 > - add sp, sp, #S_FRAME_SIZE - S_PC > - movs pc, lr @ return & move spsr_svc into cpsr > + slow_restore_user_regs > > /* > * This is how we return from a fork. > */ > ENTRY(ret_from_fork) > +#ifdef CONFIG_IPIPE > + enable_irq > +#endif /* CONFIG_IPIPE */ > bl schedule_tail > get_thread_info tsk > ldr r1, [tsk, #TI_FLAGS] @ check for syscall tracing > @@ -197,8 +206,18 @@ ENTRY(vector_swi) > bic scno, scno, #0xff000000 @ mask off SWI op-code > eor scno, scno, #__NR_SYSCALL_BASE @ check OS number > #endif > - > stmdb sp!, {r4, r5} @ push fifth and sixth args > +#ifdef CONFIG_IPIPE > + stmfd sp!, {r0-r3, ip} > + add r1, sp, #S_OFF > + add r1, r1, #20 > + mov r0, scno > + bl __ipipe_syscall_root > + cmp r0, #0 > + ldmfd sp!, {r0-r3, ip} > + blt __ipipe_ret_fast_syscall > + bgt __ipipe_fast_exit_syscall > +#endif /* CONFIG_IPIPE */ > tst ip, #_TIF_SYSCALL_TRACE @ are we tracing syscalls? > bne __sys_trace > > @@ -245,6 +264,9 @@ __sys_trace_return: > __cr_alignment: > .word cr_alignment > #endif > +#ifdef CONFIG_IPIPE > + .word __ipipe_syscall_root > +#endif > .ltorg > > /* > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/entry-header.S linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-header.S > --- linux-2.6.16.5-tcl1/arch/arm/kernel/entry-header.S 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/entry-header.S 2006-07-16 15:01:24.000000000 +0200 > @@ -68,6 +68,15 @@ > #endif > .endm > > + .macro slow_restore_user_regs > + ldr r1, [sp, #S_PSR] @ get calling cpsr > + ldr lr, [sp, #S_PC]! @ get pc > + msr spsr_cxsf, r1 @ save in spsr_svc > + ldmdb sp, {r0 - lr}^ @ get calling r1 - lr > + mov r0, r0 > + add sp, sp, #S_FRAME_SIZE - S_PC > + movs pc, lr @ return & move spsr_svc into cpsr > + .endm > > /* > * These are the registers used in the syscall handler, and allow us to > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-core.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-core.c > --- linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-core.c 1970-01-01 01:00:00.000000000 +0100 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-core.c 2006-07-16 15:01:24.000000000 +0200 > @@ -0,0 +1,239 @@ > +/* -*- linux-c -*- > + * linux/arch/arm/kernel/ipipe-core.c > + * > + * Copyright (C) 2002-2005 Philippe Gerum. > + * Copyright (C) 2004 Wolfgang Grandegger (Adeos/arm port over 2.4). > + * Copyright (C) 2005 Heikki Lindholm (PowerPC 970 fixes). > + * Copyright (C) 2005 Stelian Pop. > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, > + * USA; either version 2 of the License, or (at your option) any later > + * version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. > + * > + * Architecture-dependent I-PIPE core support for ARM. > + */ > + > +#include <linux/config.h> > +#include <linux/kernel.h> > +#include <linux/smp.h> > +#include <linux/sched.h> > +#include <linux/slab.h> > +#include <linux/bitops.h> > +#include <linux/interrupt.h> > +#include <linux/module.h> > +#include <asm/system.h> > +#include <asm/atomic.h> > +#include <asm/hardirq.h> > +#include <asm/io.h> > + > +/* Current reload value for the decrementer. */ > +unsigned long __ipipe_decr_ticks; > + > +/* Next tick date (timebase value). */ > +unsigned long long __ipipe_decr_next[IPIPE_NR_CPUS]; > + > +struct pt_regs __ipipe_tick_regs[IPIPE_NR_CPUS]; > + > +#ifdef CONFIG_SMP > + > +static cpumask_t __ipipe_cpu_sync_map; > + > +static cpumask_t __ipipe_cpu_lock_map; > + > +static ipipe_spinlock_t __ipipe_cpu_barrier = IPIPE_SPIN_LOCK_UNLOCKED; > + > +static atomic_t __ipipe_critical_count = ATOMIC_INIT(0); > + > +static void (*__ipipe_cpu_sync) (void); > + > +/* Always called with hw interrupts off. */ > + > +void __ipipe_do_critical_sync(unsigned irq) > +{ > + ipipe_declare_cpuid; > + > + ipipe_load_cpuid(); > + > + cpu_set(cpuid, __ipipe_cpu_sync_map); > + > + /* > + * Now we are in sync with the lock requestor running on another > + * CPU. Enter a spinning wait until he releases the global > + * lock. > + */ > + spin_lock_hw(&__ipipe_cpu_barrier); > + > + /* Got it. Now get out. */ > + > + if (__ipipe_cpu_sync) > + /* Call the sync routine if any. */ > + __ipipe_cpu_sync(); > + > + spin_unlock_hw(&__ipipe_cpu_barrier); > + > + cpu_clear(cpuid, __ipipe_cpu_sync_map); > +} > + > +#endif /* CONFIG_SMP */ > + > +/* > + * ipipe_critical_enter() -- Grab the superlock excluding all CPUs > + * but the current one from a critical section. This lock is used when > + * we must enforce a global critical section for a single CPU in a > + * possibly SMP system whichever context the CPUs are running. > + */ > +unsigned long ipipe_critical_enter(void (*syncfn) (void)) > +{ > + unsigned long flags; > + > + local_irq_save_hw(flags); > + > +#ifdef CONFIG_SMP > + if (num_online_cpus() > 1) { /* We might be running a SMP-kernel on a UP box... */ > + ipipe_declare_cpuid; > + cpumask_t lock_map; > + > + ipipe_load_cpuid(); > + > + if (!cpu_test_and_set(cpuid, __ipipe_cpu_lock_map)) { > + while (cpu_test_and_set(BITS_PER_LONG - 1, > + __ipipe_cpu_lock_map)) { > + int n = 0; > + do { > + cpu_relax(); > + } while (++n < cpuid); > + } > + > + spin_lock_hw(&__ipipe_cpu_barrier); > + > + __ipipe_cpu_sync = syncfn; > + > + /* Send the sync IPI to all processors but the current one. */ > + send_IPI_allbutself(IPIPE_CRITICAL_VECTOR); > + > + cpus_andnot(lock_map, cpu_online_map, > + __ipipe_cpu_lock_map); > + > + while (!cpus_equal(__ipipe_cpu_sync_map, lock_map)) > + cpu_relax(); > + } > + > + atomic_inc(&__ipipe_critical_count); > + } > +#endif /* CONFIG_SMP */ > + > + return flags; > +} > + > +/* ipipe_critical_exit() -- Release the superlock. */ > + > +void ipipe_critical_exit(unsigned long flags) > +{ > +#ifdef CONFIG_SMP > + if (num_online_cpus() > 1) { /* We might be running a SMP-kernel on a UP box... */ > + ipipe_declare_cpuid; > + > + ipipe_load_cpuid(); > + > + if (atomic_dec_and_test(&__ipipe_critical_count)) { > + spin_unlock_hw(&__ipipe_cpu_barrier); > + > + while (!cpus_empty(__ipipe_cpu_sync_map)) > + cpu_relax(); > + > + cpu_clear(cpuid, __ipipe_cpu_lock_map); > + cpu_clear(BITS_PER_LONG - 1, __ipipe_cpu_lock_map); > + } > + } > +#endif /* CONFIG_SMP */ > + > + local_irq_restore_hw(flags); > +} > + > +void __ipipe_init_platform(void) > +{ > + __ipipe_decr_ticks = __ipipe_mach_ticks_per_jiffy; > +} > + > +int ipipe_get_sysinfo(struct ipipe_sysinfo *info) > +{ > + info->ncpus = num_online_cpus(); > + info->cpufreq = ipipe_cpu_freq(); > + info->archdep.tmirq = __ipipe_mach_timerint; > + info->archdep.tmfreq = info->cpufreq; > + > + return 0; > +} > + > +/* > + * ipipe_trigger_irq() -- Push the interrupt at front of the pipeline > + * just like if it has been actually received from a hw source. Also > + * works for virtual interrupts. > + */ > +int ipipe_trigger_irq(unsigned irq) > +{ > + unsigned long flags; > + > + if (irq >= IPIPE_NR_IRQS || > + (ipipe_virtual_irq_p(irq) > + && !test_bit(irq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map))) > + return -EINVAL; > + > + local_irq_save_hw(flags); > + > + __ipipe_handle_irq(irq, NULL); > + > + local_irq_restore_hw(flags); > + > + return 1; > +} > + > +static void __ipipe_set_decr(void) > +{ > + ipipe_declare_cpuid; > + > + ipipe_load_cpuid(); > + > + __ipipe_decr_next[cpuid] = __ipipe_read_timebase() + __ipipe_decr_ticks; > + __ipipe_mach_set_dec(__ipipe_decr_ticks); > +} > + > +int ipipe_tune_timer(unsigned long ns, int flags) > +{ > + unsigned long x, ticks; > + > + if (flags & IPIPE_RESET_TIMER) > + ticks = __ipipe_mach_ticks_per_jiffy; > + else { > + ticks = (ns / 1000) * (__ipipe_mach_ticks_per_jiffy) / (1000000 / HZ); > + > + if (ticks > __ipipe_mach_ticks_per_jiffy) > + return -EINVAL; > + } > + > + x = ipipe_critical_enter(&__ipipe_set_decr); /* Sync with all CPUs */ > + __ipipe_decr_ticks = ticks; > + __ipipe_set_decr(); > + ipipe_critical_exit(x); > + > + return 0; > +} > + > +EXPORT_SYMBOL(__ipipe_decr_ticks); > +EXPORT_SYMBOL(__ipipe_decr_next); > +EXPORT_SYMBOL(ipipe_critical_enter); > +EXPORT_SYMBOL(ipipe_critical_exit); > +EXPORT_SYMBOL(ipipe_trigger_irq); > +EXPORT_SYMBOL(ipipe_get_sysinfo); > +EXPORT_SYMBOL(ipipe_tune_timer); > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-root.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c > --- linux-2.6.16.5-tcl1/arch/arm/kernel/ipipe-root.c 1970-01-01 01:00:00.000000000 +0100 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c 2006-07-17 14:59:06.000000000 +0200 > @@ -0,0 +1,379 @@ > +/* -*- linux-c -*- > + * linux/arch/arm/kernel/ipipe-root.c > + * > + * Copyright (C) 2002-2005 Philippe Gerum (Adeos/ppc port over 2.6). > + * Copyright (C) 2004 Wolfgang Grandegger (Adeos/ppc port over 2.4). > + * Copyright (C) 2005 Stelian Pop. > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, > + * USA; either version 2 of the License, or (at your option) any later > + * version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. > + * > + * Architecture-dependent I-pipe support for ARM. > + */ > + > +#include <linux/config.h> > +#include <linux/kernel.h> > +#include <linux/sched.h> > +#include <linux/module.h> > +#include <linux/slab.h> > +#include <linux/interrupt.h> > +#include <linux/errno.h> > +#include <asm/system.h> > +#include <asm/hardirq.h> > +#include <asm/atomic.h> > +#include <asm/io.h> > +#include <asm/irq.h> > +#include <asm/unistd.h> > +#include <asm/mach/irq.h> > +#include <asm/mmu_context.h> > + > +extern struct irqdesc irq_desc[]; > +extern spinlock_t irq_controller_lock; > +extern asmlinkage void asm_do_IRQ(unsigned int irq, struct pt_regs *regs); > + > +static struct irqchip __ipipe_std_irq_dtype[NR_IRQS]; > + > +static void __ipipe_override_irq_unmask(unsigned irq) > +{ > + unsigned long flags; > + > + local_irq_save_hw(flags); > + ipipe_irq_unlock(irq); > + __ipipe_std_irq_dtype[irq].unmask(irq); > + local_irq_restore_hw(flags); > +} > + > +static void __ipipe_override_irq_mask(unsigned irq) > +{ > + unsigned long flags; > + > + local_irq_save_hw(flags); > + ipipe_irq_lock(irq); > + __ipipe_std_irq_dtype[irq].mask(irq); > + local_irq_restore_hw(flags); > +} > + > +static void __ipipe_override_irq_mask_ack(unsigned irq) > +{ > + unsigned long flags; > + > + local_irq_save_hw(flags); > + ipipe_irq_lock(irq); > + __ipipe_std_irq_dtype[irq].ack(irq); > + local_irq_restore_hw(flags); > +} > + > + > +static void __ipipe_enable_sync(void) > +{ > + __ipipe_decr_next[ipipe_processor_id()] = > + __ipipe_read_timebase() + __ipipe_mach_get_dec(); > +} > + > +/* > + * __ipipe_enable_pipeline() -- We are running on the boot CPU, hw > + * interrupts are off, and secondary CPUs are still lost in space. > + */ > +void __ipipe_enable_pipeline(void) > +{ > + unsigned long flags; > + unsigned irq; > + > + flags = ipipe_critical_enter(&__ipipe_enable_sync); > + > + /* First, virtualize all interrupts from the root domain. */ > + > + for (irq = 0; irq < NR_IRQS; irq++) > + ipipe_virtualize_irq(ipipe_root_domain, > + irq, > + (ipipe_irq_handler_t)&asm_do_IRQ, NULL, > + (irq == __ipipe_mach_timerint) ? &__ipipe_ack_timerirq : &__ipipe_ack_irq, > + IPIPE_HANDLE_MASK | IPIPE_PASS_MASK); > + > + /* > + * Interpose on the IRQ control routines so we can make them > + * atomic using hw masking and prevent the interrupt log from > + * being untimely flushed. > + */ > + > + for (irq = 0; irq < NR_IRQS; irq++) > + __ipipe_std_irq_dtype[irq] = *irq_desc[irq].chip; > + > + /* > + * The original controller structs are often shared, so we first > + * save them all before changing any of them. Notice that we don't > + * override the ack() handler since we will enforce the necessary > + * setup in __ipipe_ack_irq(). > + */ > + > + for (irq = 0; irq < NR_IRQS; irq++) { > + if (irq_desc[irq].chip->mask != NULL) > + irq_desc[irq].chip->mask = __ipipe_override_irq_mask; > + > + if (irq_desc[irq].chip->unmask != NULL) > + irq_desc[irq].chip->unmask = __ipipe_override_irq_unmask; > + > + if (irq_desc[irq].chip->ack != NULL) > + irq_desc[irq].chip->ack = __ipipe_override_irq_mask_ack; > + } > + > + __ipipe_decr_next[ipipe_processor_id()] = > + __ipipe_read_timebase() + __ipipe_mach_get_dec(); > + > + ipipe_critical_exit(flags); > +} > + > +int __ipipe_ack_irq(unsigned irq) > +{ > + unsigned long flags; > + ipipe_declare_cpuid; > + > + /* > + * No need to mask IRQs at hw level: we are always called from > + * __ipipe_handle_irq(), so interrupts are already off. We > + * stall the pipeline so that spin_lock_irq*() ops won't > + * unintentionally flush it, since this could cause infinite > + * recursion. > + */ > + > + ipipe_load_cpuid(); > + flags = ipipe_test_and_stall_pipeline(); > + preempt_disable(); > + spin_lock_hw(&irq_controller_lock); > + __ipipe_std_irq_dtype[irq].ack(irq); > + spin_unlock_hw(&irq_controller_lock); > + preempt_enable_no_resched(); > + ipipe_restore_pipeline_nosync(ipipe_percpu_domain[cpuid], flags, cpuid); > + > + return 1; > +} > + > +int __ipipe_ack_timerirq(unsigned irq) > +{ > + unsigned long flags; > + ipipe_declare_cpuid; > + > + ipipe_load_cpuid(); > + flags = ipipe_test_and_stall_pipeline(); > + preempt_disable(); > + spin_lock_hw(&irq_controller_lock); > + __ipipe_mach_acktimer(); > + __ipipe_std_irq_dtype[irq].ack(irq); > + __ipipe_std_irq_dtype[irq].unmask(irq); > + spin_unlock_hw(&irq_controller_lock); > + preempt_enable_no_resched(); > + ipipe_restore_pipeline_nosync(ipipe_percpu_domain[cpuid], flags, cpuid); > + > + return 1; > +} > + > +/* > + * __ipipe_handle_irq() -- IPIPE's generic IRQ handler. An optimistic > + * interrupt protection log is maintained here for each domain. Hw > + * interrupts are off on entry. > + */ > +void __ipipe_handle_irq(int irq, struct pt_regs *regs) > +{ > + struct ipipe_domain *this_domain, *next_domain; > + struct list_head *head, *pos; > + ipipe_declare_cpuid; > + int m_ack, s_ack; > + > + m_ack = (regs == NULL); > + > + if (irq >= IPIPE_NR_IRQS) { > + printk(KERN_ERR "I-pipe: spurious interrupt %d\n", irq); > + return; > + } > + > + ipipe_load_cpuid(); > + > + this_domain = ipipe_percpu_domain[cpuid]; > + > + if (test_bit(IPIPE_STICKY_FLAG, &this_domain->irqs[irq].control)) > + head = &this_domain->p_link; > + else { > + head = __ipipe_pipeline.next; > + next_domain = list_entry(head, struct ipipe_domain, p_link); > + if (likely(test_bit(IPIPE_WIRED_FLAG, &next_domain->irqs[irq].control))) { > + if (!m_ack && next_domain->irqs[irq].acknowledge != NULL) > + next_domain->irqs[irq].acknowledge(irq); > + if (likely(__ipipe_dispatch_wired(next_domain, irq))) > + goto finalize; > + return; > + } > + } > + > + /* Ack the interrupt. */ > + > + s_ack = m_ack; > + pos = head; > + > + while (pos != &__ipipe_pipeline) { > + next_domain = list_entry(pos, struct ipipe_domain, p_link); > + > + /* > + * For each domain handling the incoming IRQ, mark it as > + * pending in its log. > + */ > + if (test_bit(IPIPE_HANDLE_FLAG, > + &next_domain->irqs[irq].control)) { > + /* > + * Domains that handle this IRQ are polled for > + * acknowledging it by decreasing priority order. The > + * interrupt must be made pending _first_ in the > + * domain's status flags before the PIC is unlocked. > + */ > + > + next_domain->cpudata[cpuid].irq_counters[irq].total_hits++; > + next_domain->cpudata[cpuid].irq_counters[irq].pending_hits++; > + __ipipe_set_irq_bit(next_domain, cpuid, irq); > + > + /* > + * Always get the first master acknowledge available. > + * Once we've got it, allow slave acknowledge > + * handlers to run (until one of them stops us). > + */ > + if (next_domain->irqs[irq].acknowledge != NULL) { > + if (!m_ack) > + m_ack = next_domain->irqs[irq].acknowledge(irq); > + else if (test_bit > + (IPIPE_SHARED_FLAG, > + &next_domain->irqs[irq].control) && !s_ack) > + s_ack = next_domain->irqs[irq].acknowledge(irq); > + } > + } > + > + /* > + * If the domain does not want the IRQ to be passed down the > + * interrupt pipe, exit the loop now. > + */ > + > + if (!test_bit(IPIPE_PASS_FLAG, &next_domain->irqs[irq].control)) > + break; > + > + pos = next_domain->p_link.next; > + } > + > +finalize: > + /* > + * Now walk the pipeline, yielding control to the highest > + * priority domain that has pending interrupt(s) or > + * immediately to the current domain if the interrupt has been > + * marked as 'sticky'. This search does not go beyond the > + * current domain in the pipeline. > + */ > + > + __ipipe_walk_pipeline(head, cpuid); > +} > + > +asmlinkage int __ipipe_grab_irq(int irq, struct pt_regs *regs) > +{ > + ipipe_declare_cpuid; > + > + if (irq == __ipipe_mach_timerint) { > + > + __ipipe_tick_regs[cpuid].ARM_cpsr = regs->ARM_cpsr; > + __ipipe_tick_regs[cpuid].ARM_pc = regs->ARM_pc; > + > + if (__ipipe_decr_ticks != __ipipe_mach_ticks_per_jiffy) { > + unsigned long long next_date, now; > + > + next_date = __ipipe_decr_next[cpuid]; > + > + while ((now = __ipipe_read_timebase()) >= next_date) > + next_date += __ipipe_decr_ticks; > + > + __ipipe_mach_set_dec(next_date - now); > + > + __ipipe_decr_next[cpuid] = next_date; > + } > + } > + > + __ipipe_handle_irq(irq, regs); > + > + ipipe_load_cpuid(); > + > + return (ipipe_percpu_domain[cpuid] == ipipe_root_domain && > + !test_bit(IPIPE_STALL_FLAG, > + &ipipe_root_domain->cpudata[cpuid].status)); > +} > + > +asmlinkage int __ipipe_check_root(struct pt_regs *regs) > +{ > + ipipe_declare_cpuid; > + /* > + * This routine is called with hw interrupts off, so no migration > + * can occur while checking the identity of the current domain. > + */ > + ipipe_load_cpuid(); > + return (ipipe_percpu_domain[cpuid] == ipipe_root_domain && > + !test_bit(IPIPE_STALL_FLAG, > + &ipipe_root_domain->cpudata[cpuid].status)); > +} > + > +asmlinkage int __ipipe_syscall_root(unsigned long scno, struct pt_regs *regs) > +{ > + ipipe_declare_cpuid; > + unsigned long flags, origr7; > + > + /* We use r7 to pass the syscall number to the other domains */ > + origr7 = regs->ARM_r7; > + regs->ARM_r7 = __NR_SYSCALL_BASE + scno; > + > + /* > + * This routine either returns: > + * 0 -- if the syscall is to be passed to Linux; > + * >0 -- if the syscall should not be passed to Linux, and no > + * tail work should be performed; > + * <0 -- if the syscall should not be passed to Linux but the > + * tail work has to be performed (for handling signals etc). > + */ > + > + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SYSCALL) && > + __ipipe_dispatch_event(IPIPE_EVENT_SYSCALL,regs) > 0) { > + /* > + * We might enter here over a non-root domain and exit > + * over the root one as a result of the syscall > + * (i.e. by recycling the register set of the current > + * context across the migration), so we need to fixup > + * the interrupt flag upon return too, so that > + * __ipipe_unstall_iret_root() resets the correct > + * stall bit on exit. > + */ > + if (ipipe_current_domain == ipipe_root_domain && !in_atomic()) { > + /* > + * Sync pending VIRQs before _TIF_NEED_RESCHED > + * is tested. > + */ > + ipipe_lock_cpu(flags); > + if ((ipipe_root_domain->cpudata[cpuid].irq_pending_hi & IPIPE_IRQMASK_VIRT) != 0) > + __ipipe_sync_pipeline(IPIPE_IRQMASK_VIRT); > + ipipe_unlock_cpu(flags); > + regs->ARM_r7 = origr7; > + return -1; > + } > + regs->ARM_r7 = origr7; > + return 1; > + } > + > + regs->ARM_r7 = origr7; > + return 0; > +} > + > +EXPORT_SYMBOL_GPL(show_stack); > +#ifndef MULTI_CPU > +EXPORT_SYMBOL_GPL(cpu_do_switch_mm); > +#endif /* MULTI_CPU */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/irq.c > --- linux-2.6.16.5-tcl1/arch/arm/kernel/irq.c 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/irq.c 2006-07-16 15:01:24.000000000 +0200 > @@ -54,10 +54,11 @@ > > static int noirqdebug; > static volatile unsigned long irq_err_count; > -static DEFINE_SPINLOCK(irq_controller_lock); > +DEFINE_SPINLOCK(irq_controller_lock); > static LIST_HEAD(irq_pending); > > struct irqdesc irq_desc[NR_IRQS]; > +EXPORT_SYMBOL(irq_desc); > void (*init_arch_irq)(void) __initdata = NULL; > > /* > @@ -412,7 +413,9 @@ do_edge_IRQ(unsigned int irq, struct irq > /* > * Acknowledge and clear the IRQ, but don't mask it. > */ > +#ifndef CONFIG_IPIPE > desc->chip->ack(irq); > +#endif /* CONFIG_IPIPE */ > > /* > * Mark the IRQ currently in progress. > @@ -450,8 +453,10 @@ do_edge_IRQ(unsigned int irq, struct irq > * currently running. Delay it. > */ > desc->pending = 1; > +#ifndef CONFIG_IPIPE > desc->chip->mask(irq); > desc->chip->ack(irq); > +#endif /* CONFIG_IPIPE */ > } > > /* > @@ -468,7 +473,9 @@ do_level_IRQ(unsigned int irq, struct ir > /* > * Acknowledge, clear _AND_ disable the interrupt. > */ > +#ifndef CONFIG_IPIPE > desc->chip->ack(irq); > +#endif /* CONFIG_IPIPE */ > > if (likely(!desc->disable_depth)) { > kstat_cpu(cpu).irqs[irq]++; > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/process.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/process.c > --- linux-2.6.16.5-tcl1/arch/arm/kernel/process.c 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/process.c 2006-07-16 15:01:24.000000000 +0200 > @@ -89,12 +89,12 @@ static void default_idle(void) > if (hlt_counter) > cpu_relax(); > else { > - local_irq_disable(); > + local_irq_disable_hw(); > if (!need_resched()) { > timer_dyn_reprogram(); > arch_idle(); > } > - local_irq_enable(); > + local_irq_enable_hw(); > } > } > > @@ -120,6 +120,7 @@ void cpu_idle(void) > > if (!idle) > idle = default_idle; > + ipipe_suspend_domain(); > leds_event(led_idle_start); > while (!need_resched()) > idle(); > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/ptrace.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ptrace.c > --- linux-2.6.16.5-tcl1/arch/arm/kernel/ptrace.c 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ptrace.c 2006-07-16 15:01:24.000000000 +0200 > @@ -486,6 +486,10 @@ void ptrace_break(struct task_struct *ts > > static int break_trap(struct pt_regs *regs, unsigned int instr) > { > + > + if (ipipe_trap_notify(IPIPE_TRAP_BREAK,regs)) > + return 0; > + > ptrace_break(current, regs); > return 0; > } > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/kernel/traps.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/traps.c > --- linux-2.6.16.5-tcl1/arch/arm/kernel/traps.c 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/traps.c 2006-07-16 15:01:24.000000000 +0200 > @@ -310,6 +310,9 @@ asmlinkage void do_undefinstr(struct pt_ > } > spin_unlock_irq(&undef_lock); > > + if (ipipe_trap_notify(IPIPE_TRAP_UNDEFINSTR,regs)) > + return; > + > #ifdef CONFIG_DEBUG_USER > if (user_debug & UDBG_UNDEFINED) { > printk(KERN_INFO "%s (%d): undefined instruction: pc=%p\n", > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-integrator/core.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/core.c > --- linux-2.6.16.5-tcl1/arch/arm/mach-integrator/core.c 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/core.c 2006-07-16 15:01:24.000000000 +0200 > @@ -2,6 +2,7 @@ > * linux/arch/arm/mach-integrator/core.c > * > * Copyright (C) 2000-2003 Deep Blue Solutions Ltd > + * Copyright (C) 2005 Stelian Pop. > * > * This program is free software; you can redistribute it and/or modify > * it under the terms of the GNU General Public License version 2, as > @@ -148,53 +149,57 @@ EXPORT_SYMBOL(cm_control); > /* > * How long is the timer interval? > */ > -#define TIMER_INTERVAL (TICKS_PER_uSEC * mSEC_10) > -#if TIMER_INTERVAL >= 0x100000 > -#define TICKS2USECS(x) (256 * (x) / TICKS_PER_uSEC) > -#elif TIMER_INTERVAL >= 0x10000 > -#define TICKS2USECS(x) (16 * (x) / TICKS_PER_uSEC) > -#else > #define TICKS2USECS(x) ((x) / TICKS_PER_uSEC) > -#endif > > static unsigned long timer_reload; > +static unsigned long timer_interval; > +static unsigned long timer_lxlost; > +static int tscok; > + > +#ifdef CONFIG_IPIPE > +int __ipipe_mach_timerint = IRQ_TIMERINT1; > +static unsigned long long __ipipe_mach_tsc; > +static DEFINE_SPINLOCK(timer_lock); > + > +int __ipipe_mach_timerstolen = 0; > +EXPORT_SYMBOL(__ipipe_mach_timerstolen); > +#endif > > /* > - * Returns number of ms since last clock interrupt. Note that interrupts > - * will have been disabled by do_gettimeoffset() > + * Called with IRQ disabled from do_gettimeofday(). > */ > -unsigned long integrator_gettimeoffset(void) > +static inline unsigned long integrator_getticksoffset(void) > { > - unsigned long ticks1, ticks2, status; > + unsigned long ticks; > > - /* > - * Get the current number of ticks. Note that there is a race > - * condition between us reading the timer and checking for > - * an interrupt. We get around this by ensuring that the > - * counter has not reloaded between our two reads. > - */ > - ticks2 = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; > - do { > - ticks1 = ticks2; > - status = __raw_readl(VA_IC_BASE + IRQ_RAW_STATUS); > - ticks2 = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; > - } while (ticks2 > ticks1); > + if (!tscok) > + return 0; > > - /* > - * Number of ticks since last interrupt. > - */ > - ticks1 = timer_reload - ticks2; > + ticks = readl(TIMER1_VA_BASE + TIMER_VALUE) & 0xffff; > > - /* > - * Interrupt pending? If so, we've reloaded once already. > - */ > - if (status & (1 << IRQ_TIMERINT1)) > - ticks1 += timer_reload; > + if (ticks > timer_reload) > + ticks = 0xffff + timer_reload - ticks; > + else > + ticks = timer_reload - ticks; > > + if (timer_interval < 0x10000) > + return ticks; > + else if (timer_interval < 0x100000) > + return ticks * 16; > + else > + return ticks * 256; > +} > + > +/* > + * Returns number of ms since last clock interrupt. Note that interrupts > + * will have been disabled by do_gettimeoffset() > + */ > +unsigned long integrator_gettimeoffset(void) > +{ > /* > * Convert the ticks to usecs > */ > - return TICKS2USECS(ticks1); > + return TICKS2USECS(timer_lxlost + integrator_getticksoffset()); > } > > /* > @@ -205,10 +210,22 @@ integrator_timer_interrupt(int irq, void > { > write_seqlock(&xtime_lock); > > + timer_lxlost = 0; > + > +#ifdef CONFIG_IPIPE > /* > - * clear the interrupt > + * If Linux is the only domain, ack the timer and reprogram it > */ > - writel(1, TIMER1_VA_BASE + TIMER_INTCLR); > + if (!__ipipe_mach_timerstolen) { > + __ipipe_mach_tsc += integrator_getticksoffset(); > +#else > + writel(1, TIMER1_VA_BASE + TIMER_INTCLR); > +#endif > + > + writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); > +#ifdef CONFIG_IPIPE > + } > +#endif > > /* > * the clock tick routines are only processed on the > @@ -239,24 +256,30 @@ static struct irqaction integrator_timer > .handler = integrator_timer_interrupt, > }; > > -/* > - * Set up timer interrupt, and return the current time in seconds. > - */ > -void __init integrator_time_init(unsigned long reload, unsigned int ctrl) > +static inline void set_dec(unsigned long reload) > { > - unsigned int timer_ctrl = TIMER_CTRL_ENABLE | TIMER_CTRL_PERIODIC; > + unsigned int ctrl = TIMER_CTRL_ENABLE | TIMER_CTRL_IE; > > timer_reload = reload; > - timer_ctrl |= ctrl; > + timer_interval = reload; > > - if (timer_reload > 0x100000) { > + if (timer_reload >= 0x100000) { > timer_reload >>= 8; > - timer_ctrl |= TIMER_CTRL_DIV256; > - } else if (timer_reload > 0x010000) { > + ctrl |= TIMER_CTRL_DIV256; > + } else if (timer_reload >= 0x010000) { > timer_reload >>= 4; > - timer_ctrl |= TIMER_CTRL_DIV16; > + ctrl |= TIMER_CTRL_DIV16; > } > > + writel(ctrl, TIMER1_VA_BASE + TIMER_CTRL); > + writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); > +} > + > +/* > + * Set up timer interrupt, and return the current time in seconds. > + */ > +void __init integrator_time_init(unsigned long reload, unsigned int ctrl) > +{ > /* > * Initialise to a known state (all timers off) > */ > @@ -264,12 +287,51 @@ void __init integrator_time_init(unsigne > writel(0, TIMER1_VA_BASE + TIMER_CTRL); > writel(0, TIMER2_VA_BASE + TIMER_CTRL); > > - writel(timer_reload, TIMER1_VA_BASE + TIMER_LOAD); > - writel(timer_reload, TIMER1_VA_BASE + TIMER_VALUE); > - writel(timer_ctrl, TIMER1_VA_BASE + TIMER_CTRL); > + set_dec(reload); > > /* > * Make irqs happen for the system timer > */ > setup_irq(IRQ_TIMERINT1, &integrator_timer_irq); > + > + tscok = 1; > } > + > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_acktimer(void) > +{ > + writel(1, TIMER1_VA_BASE + TIMER_INTCLR); > +} > + > +unsigned long long __ipipe_mach_get_tsc(void) > +{ > + unsigned long long result; > + unsigned long flags; > + > + spin_lock_irqsave_hw(&timer_lock, flags); > + result = __ipipe_mach_tsc + integrator_getticksoffset(); > + spin_unlock_irqrestore_hw(&timer_lock, flags); > + return result; > +} > +EXPORT_SYMBOL(__ipipe_mach_get_tsc); > + > +void __ipipe_mach_set_dec(unsigned long reload) > +{ > + unsigned long ticks; > + unsigned long flags; > + > + spin_lock_irqsave_hw(&timer_lock, flags); > + ticks = integrator_getticksoffset(); > + __ipipe_mach_tsc += ticks; > + timer_lxlost += ticks; > + > + set_dec(reload); > + spin_unlock_irqrestore_hw(&timer_lock, flags); > +} > +EXPORT_SYMBOL(__ipipe_mach_set_dec); > + > +unsigned long __ipipe_mach_get_dec(void) > +{ > + return readl(TIMER1_VA_BASE + TIMER_VALUE); > +} > +#endif /* CONFIG_IPIPE */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-integrator/integrator_cp.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c > --- linux-2.6.16.5-tcl1/arch/arm/mach-integrator/integrator_cp.c 2006-05-07 16:41:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c 2006-07-16 15:01:24.000000000 +0200 > @@ -2,6 +2,7 @@ > * linux/arch/arm/mach-integrator/integrator_cp.c > * > * Copyright (C) 2003 Deep Blue Solutions Ltd > + * Copyright (C) 2005 Stelian Pop. > * > * This program is free software; you can redistribute it and/or modify > * it under the terms of the GNU General Public License as published by > @@ -568,9 +569,14 @@ static void __init intcp_init(void) > > #define TIMER_CTRL_IE (1 << 5) /* Interrupt Enable */ > > +#ifdef CONFIG_IPIPE > +unsigned int __ipipe_mach_ticks_per_jiffy = 1000000 * TICKS_PER_uSEC / HZ; > +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); > +#endif > + > static void __init intcp_timer_init(void) > { > - integrator_time_init(1000000 / HZ, TIMER_CTRL_IE); > + integrator_time_init(1000000 * TICKS_PER_uSEC / HZ, TIMER_CTRL_IE); > } > > static struct sys_timer cp_timer = { > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-pxa/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/arch/arm/mm/fault.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mm/fault.c > --- linux-2.6.16.5-tcl1/arch/arm/mm/fault.c 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mm/fault.c 2006-07-16 15:01:24.000000000 +0200 > @@ -223,6 +223,9 @@ do_page_fault(unsigned long addr, unsign > struct mm_struct *mm; > int fault, sig, code; > > + if (ipipe_trap_notify(IPIPE_TRAP_ACCESS,regs)) > + return 0; > + > tsk = current; > mm = tsk->mm; > > @@ -354,6 +357,9 @@ do_translation_fault(unsigned long addr, > bad_area: > tsk = current; > > + if (ipipe_trap_notify(IPIPE_TRAP_ACCESS,regs)) > + return 0; > + > do_bad_area(tsk, tsk->active_mm, addr, fsr, regs); > return 0; > } > @@ -366,6 +372,10 @@ static int > do_sect_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) > { > struct task_struct *tsk = current; > + > + if (ipipe_trap_notify(IPIPE_TRAP_SECTION,regs)) > + return 0; > + > do_bad_area(tsk, tsk->active_mm, addr, fsr, regs); > return 0; > } > @@ -376,6 +386,9 @@ do_sect_fault(unsigned long addr, unsign > static int > do_bad(unsigned long addr, unsigned int fsr, struct pt_regs *regs) > { > + if (ipipe_trap_notify(IPIPE_TRAP_DABT,regs)) > + return 0; > + > return 1; > } > > @@ -451,6 +464,9 @@ do_DataAbort(unsigned long addr, unsigne > if (!inf->fn(addr, fsr, regs)) > return; > > + if (ipipe_trap_notify(IPIPE_TRAP_UNKNOWN,regs)) > + return; > + > printk(KERN_ALERT "Unhandled fault: %s (0x%03x) at 0x%08lx\n", > inf->name, fsr, addr); > > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/drivers/pci/msi.c linux-2.6.16.5-tcl1-ipipe/drivers/pci/msi.c > --- linux-2.6.16.5-tcl1/drivers/pci/msi.c 2006-05-07 16:41:40.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/drivers/pci/msi.c 2006-07-16 15:01:24.000000000 +0200 > @@ -151,6 +151,21 @@ static void unmask_MSI_irq(unsigned int > msi_set_mask_bit(vector, 0); > } > > +#ifdef CONFIG_IPIPE > +static void ack_MSI_irq_w_maskbit(unsigned int vector) > +{ > + mask_MSI_irq(vector); > + __ack_APIC_irq(); > +} > +static void ack_MSI_irq_wo_maskbit(unsigned int vector) > +{ > + __ack_APIC_irq(); > +} > +#else /* !CONFIG_IPIPE */ > +#define ack_MSI_irq_wo_maskbit do_nothing > +#define ack_MSI_irq_w_maskbit mask_MSI_irq > +#endif /* CONFIG_IPIPE */ > + > static unsigned int startup_msi_irq_wo_maskbit(unsigned int vector) > { > struct msi_desc *entry; > @@ -214,7 +229,7 @@ static struct hw_interrupt_type msix_irq > .shutdown = shutdown_msi_irq, > .enable = unmask_MSI_irq, > .disable = mask_MSI_irq, > - .ack = mask_MSI_irq, > + .ack = ack_MSI_irq_w_maskbit, > .end = end_msi_irq_w_maskbit, > .set_affinity = set_msi_affinity > }; > @@ -230,7 +245,7 @@ static struct hw_interrupt_type msi_irq_ > .shutdown = shutdown_msi_irq, > .enable = unmask_MSI_irq, > .disable = mask_MSI_irq, > - .ack = mask_MSI_irq, > + .ack = ack_MSI_irq_w_maskbit, > .end = end_msi_irq_w_maskbit, > .set_affinity = set_msi_affinity > }; > @@ -246,7 +261,7 @@ static struct hw_interrupt_type msi_irq_ > .shutdown = shutdown_msi_irq, > .enable = do_nothing, > .disable = do_nothing, > - .ack = do_nothing, > + .ack = ack_MSI_irq_wo_maskbit, > .end = end_msi_irq_wo_maskbit, > .set_affinity = set_msi_affinity > }; > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/entry-macro.S linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/entry-macro.S > --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/entry-macro.S 2006-05-07 16:42:04.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/entry-macro.S 2006-07-16 15:01:24.000000000 +0200 > @@ -22,7 +22,11 @@ > teq \irqstat, #0 > ldreq \irqstat, [\base, #(INTEGRATOR_HDR_IC_OFFSET+IRQ_STATUS)] > moveq \irqnr, #IRQ_CIC_START > - > +#ifdef CONFIG_IPIPE > + tst \irqstat, #0x00000040 @ check IRQ_TIMERINT1 first > + movne \irqnr, #6 > + bne 1003f > +#endif /* CONFIG_IPIPE */ > 1001: tst \irqstat, #15 > bne 1002f > add \irqnr, \irqnr, #4 > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/platform.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/platform.h > --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/platform.h 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/platform.h 2006-07-16 15:01:24.000000000 +0200 > @@ -26,13 +26,15 @@ > * NOTE: This is a multi-hosted header file for use with uHAL and > * supported debuggers. > * > - * $Id: platform.s,v 1.32 2000/02/18 10:51:39 asims Exp $ > + * $Id: platform.h,v 1.2 2006/02/20 13:54:22 rpm Exp $ > * > * ***********************************************************************/ > > #ifndef __address_h > #define __address_h 1 > > +#include <linux/config.h> > + > /* ======================================================================== > * Integrator definitions > * ======================================================================== > @@ -436,7 +438,7 @@ > * Timer definitions > * > * Only use timer 1 & 2 > - * (both run at 24MHz and will need the clock divider set to 16). > + * (both run at 1MHZ on /CP and at 24MHz on /AP) > * > * Timer 0 runs at bus frequency and therefore could vary and currently > * uHAL can't handle that. > @@ -449,7 +451,12 @@ > > #define MAX_TIMER 2 > #define MAX_PERIOD 699050 > + > +#ifdef CONFIG_ARCH_INTEGRATOR_CP > +#define TICKS_PER_uSEC 1 > +#else > #define TICKS_PER_uSEC 24 > +#endif > > /* > * These are useconds NOT ticks. > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/timex.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/timex.h > --- linux-2.6.16.5-tcl1/include/asm-arm/arch-integrator/timex.h 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/timex.h 2006-07-16 15:01:24.000000000 +0200 > @@ -21,6 +21,6 @@ > */ > > /* > - * ?? > + * Timer rate > */ > -#define CLOCK_TICK_RATE (50000000 / 16) > +#define CLOCK_TICK_RATE (1000000) > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/atomic.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/atomic.h > --- linux-2.6.16.5-tcl1/include/asm-arm/atomic.h 2006-05-07 16:42:05.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/atomic.h 2006-07-16 15:01:24.000000000 +0200 > @@ -129,10 +129,10 @@ static inline int atomic_add_return(int > unsigned long flags; > int val; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > val = v->counter; > v->counter = val += i; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > > return val; > } > @@ -142,10 +142,10 @@ static inline int atomic_sub_return(int > unsigned long flags; > int val; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > val = v->counter; > v->counter = val -= i; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > > return val; > } > @@ -155,11 +155,11 @@ static inline int atomic_cmpxchg(atomic_ > int ret; > unsigned long flags; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > ret = v->counter; > if (likely(ret == old)) > v->counter = new; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > > return ret; > } > @@ -168,9 +168,9 @@ static inline void atomic_clear_mask(uns > { > unsigned long flags; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > *addr &= ~mask; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > } > > #endif /* __LINUX_ARM_ARCH__ */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/bitops.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/bitops.h > --- linux-2.6.16.5-tcl1/include/asm-arm/bitops.h 2006-05-07 16:42:05.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/bitops.h 2006-07-16 15:01:24.000000000 +0200 > @@ -37,9 +37,9 @@ static inline void ____atomic_set_bit(un > > p += bit >> 5; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > *p |= mask; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > } > > static inline void ____atomic_clear_bit(unsigned int bit, volatile unsigned long *p) > @@ -49,9 +49,9 @@ static inline void ____atomic_clear_bit( > > p += bit >> 5; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > *p &= ~mask; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > } > > static inline void ____atomic_change_bit(unsigned int bit, volatile unsigned long *p) > @@ -61,9 +61,9 @@ static inline void ____atomic_change_bit > > p += bit >> 5; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > *p ^= mask; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > } > > static inline int > @@ -75,10 +75,10 @@ ____atomic_test_and_set_bit(unsigned int > > p += bit >> 5; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > res = *p; > *p = res | mask; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > > return res & mask; > } > @@ -92,10 +92,10 @@ ____atomic_test_and_clear_bit(unsigned i > > p += bit >> 5; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > res = *p; > *p = res & ~mask; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > > return res & mask; > } > @@ -109,10 +109,10 @@ ____atomic_test_and_change_bit(unsigned > > p += bit >> 5; > > - local_irq_save(flags); > + local_irq_save_hw(flags); > res = *p; > *p = res ^ mask; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > > return res & mask; > } > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h > --- linux-2.6.16.5-tcl1/include/asm-arm/ipipe.h 1970-01-01 01:00:00.000000000 +0100 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h 2006-07-16 15:01:24.000000000 +0200 > @@ -0,0 +1,193 @@ > +/* -*- linux-c -*- > + * include/asm-arm/ipipe.h > + * > + * Copyright (C) 2002-2005 Philippe Gerum. > + * Copyright (C) 2005 Stelian Pop. > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, > + * USA; either version 2 of the License, or (at your option) any later > + * version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. > + */ > + > +#ifndef __ARM_IPIPE_H > +#define __ARM_IPIPE_H > + > +#include <linux/config.h> > + > +#ifdef CONFIG_IPIPE > + > +#include <linux/list.h> > +#include <linux/cpumask.h> > +#include <linux/threads.h> > +#include <asm/ptrace.h> > +#include <asm/irq.h> > +#include <asm/bitops.h> > +#include <asm/mach/irq.h> > + > +#define IPIPE_ARCH_STRING "1.3-04" > +#define IPIPE_MAJOR_NUMBER 1 > +#define IPIPE_MINOR_NUMBER 3 > +#define IPIPE_PATCH_NUMBER 4 > + > +#define IPIPE_NR_XIRQS NR_IRQS > +#define IPIPE_IRQ_ISHIFT 5 /* 25 for 32bits arch. */ > + > +#ifdef CONFIG_SMP > +#error "I-pipe/arm: SMP not yet implemented" > +#define ipipe_processor_id() (current_thread_info()->cpu) > +#else /* !CONFIG_SMP */ > +#define ipipe_processor_id() 0 > +#endif /* CONFIG_SMP */ > + > +/* Note that we disable the interrupts around context_switch, > + * or we'll get into severe problems when scheduling Xenoma > + * user space real time threads. > + * This can however cause high latencies, see for example: > + * http://www.ussg.iu.edu/hypermail/linux/kernel/0405.2/1388.html > + * This may need further optimization... > + */ > +#define prepare_arch_switch(next) \ > +do { \ > + __ipipe_dispatch_event(IPIPE_EVENT_SCHEDULE,next); \ > + local_irq_disable_hw(); \ > +} while(0) > + > +#define task_hijacked(p) \ > + ( { \ > + int x = ipipe_current_domain != ipipe_root_domain; \ > + __clear_bit(IPIPE_SYNC_FLAG, \ > + &ipipe_root_domain->cpudata[task_cpu(p)].status); \ > + local_irq_enable_hw(); \ > + x; \ > + } ) > + > +/* ARM traps */ > +#define IPIPE_TRAP_ACCESS 0 /* Data or instruction access exception */ > +#define IPIPE_TRAP_SECTION 1 /* Section fault */ > +#define IPIPE_TRAP_DABT 2 /* Generic data abort */ > +#define IPIPE_TRAP_UNKNOWN 3 /* Unknown exception */ > +#define IPIPE_TRAP_BREAK 4 /* Instruction breakpoint */ > +#define IPIPE_TRAP_FPU 5 /* Floating point exception */ > +#define IPIPE_TRAP_VFP 6 /* VFP floating point exception */ > +#define IPIPE_TRAP_UNDEFINSTR 7 /* Undefined instruction */ > +#define IPIPE_NR_FAULTS 8 > + > +/* Pseudo-vectors used for kernel events */ > +#define IPIPE_FIRST_EVENT IPIPE_NR_FAULTS > +#define IPIPE_EVENT_SYSCALL (IPIPE_FIRST_EVENT) > +#define IPIPE_EVENT_SCHEDULE (IPIPE_FIRST_EVENT + 1) > +#define IPIPE_EVENT_SIGWAKE (IPIPE_FIRST_EVENT + 2) > +#define IPIPE_EVENT_SETSCHED (IPIPE_FIRST_EVENT + 3) > +#define IPIPE_EVENT_EXIT (IPIPE_FIRST_EVENT + 4) > +#define IPIPE_EVENT_CLEANUP (IPIPE_FIRST_EVENT + 5) > +#define IPIPE_LAST_EVENT IPIPE_EVENT_CLEANUP > +#define IPIPE_NR_EVENTS (IPIPE_LAST_EVENT + 1) > + > +struct ipipe_domain; > + > +struct ipipe_sysinfo { > + > + int ncpus; /* Number of CPUs on board */ > + u64 cpufreq; /* CPU frequency (in Hz) */ > + > + /* Arch-dependent block */ > + > + struct { > + unsigned tmirq; /* Timer tick IRQ */ > + u64 tmfreq; /* Timer frequency */ > + } archdep; > +}; > + > +/* arch specific stuff */ > +extern int __ipipe_mach_timerint; > +extern int __ipipe_mach_timerstolen; > +extern unsigned int __ipipe_mach_ticks_per_jiffy; > +extern void __ipipe_mach_acktimer(void); > +extern unsigned long long __ipipe_mach_get_tsc(void); > +extern void __ipipe_mach_set_dec(unsigned long); > +extern unsigned long __ipipe_mach_get_dec(void); > + > +#define ipipe_read_tsc(t) do { t = __ipipe_mach_get_tsc(); } while (0) > +#define __ipipe_read_timebase() __ipipe_mach_get_tsc() > + > +#define ipipe_cpu_freq() (HZ * __ipipe_mach_ticks_per_jiffy) > +#define ipipe_tsc2ns(t) (((t) * 1000) / (ipipe_cpu_freq() / 1000000)) > + > +/* Private interface -- Internal use only */ > + > +#define __ipipe_check_platform() do { } while(0) > + > +#define __ipipe_enable_irq(irq) irq_desc[irq].chip->unmask(irq) > + > +#define __ipipe_disable_irq(irq) irq_desc[irq].chip->mask(irq) > + > +void __ipipe_init_platform(void); > + > +void __ipipe_enable_pipeline(void); > + > +int __ipipe_ack_irq(unsigned irq); > + > +int __ipipe_ack_timerirq(unsigned irq); > + > +void __ipipe_do_IRQ(int irq, > + struct pt_regs *regs); > + > +void __ipipe_do_timer(int irq, > + struct pt_regs *regs); > + > +void __ipipe_do_critical_sync(unsigned irq, > + void *cookie); > + > +extern unsigned long __ipipe_decr_ticks; > + > +extern unsigned long long __ipipe_decr_next[]; > + > +extern struct pt_regs __ipipe_tick_regs[]; > + > +void __ipipe_handle_irq(int irq, > + struct pt_regs *regs); > + > +#define __ipipe_tick_irq ipipe_timerint > + > +static inline unsigned long __ipipe_ffnz(unsigned long ul) > +{ > + return ffs(ul) - 1; > +} > + > +#define __ipipe_run_isr(ipd, irq, cpuid) \ > +do { \ > + if (ipd == ipipe_root_domain) { \ > + /* \ > + * Linux handlers are called w/ hw interrupts on so \ > + * that they could not defer interrupts for higher \ > + * priority domains. \ > + */ \ > + local_irq_enable_hw(); \ > + ((void (*)(unsigned, struct pt_regs *)) \ > + ipd->irqs[irq].handler) (irq, __ipipe_tick_regs + cpuid); \ > + local_irq_disable_hw(); \ > + } else { \ > + __clear_bit(IPIPE_SYNC_FLAG, &cpudata->status); \ > + ipd->irqs[irq].handler(irq,ipd->irqs[irq].cookie); \ > + __set_bit(IPIPE_SYNC_FLAG, &cpudata->status); \ > + } \ > +} while(0) > + > +#else /* !CONFIG_IPIPE */ > + > +#define task_hijacked(p) 0 > + > +#endif /* CONFIG_IPIPE */ > + > +#endif /* !__ARM_IPIPE_H */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/mmu_context.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/mmu_context.h > --- linux-2.6.16.5-tcl1/include/asm-arm/mmu_context.h 2006-05-07 15:36:58.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/mmu_context.h 2006-07-16 15:01:24.000000000 +0200 > @@ -82,14 +82,17 @@ static inline void > switch_mm(struct mm_struct *prev, struct mm_struct *next, > struct task_struct *tsk) > { > - unsigned int cpu = smp_processor_id(); > + unsigned int cpu = smp_processor_id_hw(); > > if (prev != next) { > + unsigned long flags; > + local_irq_save_hw_cond(flags); > cpu_set(cpu, next->cpu_vm_mask); > check_context(next); > cpu_switch_mm(next->pgd, next); > if (cache_is_vivt()) > cpu_clear(cpu, prev->cpu_vm_mask); > + local_irq_restore_hw_cond(flags); > } > } > > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/pgalloc.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/pgalloc.h > --- linux-2.6.16.5-tcl1/include/asm-arm/pgalloc.h 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/pgalloc.h 2006-07-16 15:01:24.000000000 +0200 > @@ -29,6 +29,11 @@ extern void free_pgd_slow(pgd_t *pgd); > > #define check_pgt_cache() do { } while (0) > > +static inline void set_pgdir(unsigned long address, pgd_t entry) > +{ > + /* nop */ > +} > + > /* > * Allocate one PTE table. > * > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/asm-arm/system.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h > --- linux-2.6.16.5-tcl1/include/asm-arm/system.h 2006-05-07 16:42:05.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h 2006-07-17 18:40:59.000000000 +0200 > @@ -186,30 +186,30 @@ static inline void sched_cacheflush(void > */ > #if __LINUX_ARM_ARCH__ >= 6 > > -#define local_irq_save(x) \ > +#define local_irq_save_hw(x) \ > ({ \ > __asm__ __volatile__( \ > - "mrs %0, cpsr @ local_irq_save\n" \ > + "mrs %0, cpsr @ local_irq_save_hw\n" \ > "cpsid i" \ > : "=r" (x) : : "memory", "cc"); \ > }) > > -#define local_irq_enable() __asm__("cpsie i @ __sti" : : : "memory", "cc") > -#define local_irq_disable() __asm__("cpsid i @ __cli" : : : "memory", "cc") > -#define local_fiq_enable() __asm__("cpsie f @ __stf" : : : "memory", "cc") > -#define local_fiq_disable() __asm__("cpsid f @ __clf" : : : "memory", "cc") > +#define local_irq_enable_hw() __asm__("cpsie i @ __sti" : : : "memory", "cc") > +#define local_irq_disable_hw() __asm__("cpsid i @ __cli" : : : "memory", "cc") > +#define local_fiq_enable_hw() __asm__("cpsie f @ __stf" : : : "memory", "cc") > +#define local_fiq_disable_hw() __asm__("cpsid f @ __clf" : : : "memory", "cc") > > #else > > /* > * Save the current interrupt enable state & disable IRQs > */ > -#define local_irq_save(x) \ > +#define local_irq_save_hw(x) \ > ({ \ > unsigned long temp; \ > (void) (&temp == &x); \ > __asm__ __volatile__( \ > - "mrs %0, cpsr @ local_irq_save\n" \ > + "mrs %0, cpsr @ local_irq_save_hw\n" \ > " orr %1, %0, #128\n" \ > " msr cpsr_c, %1" \ > : "=r" (x), "=r" (temp) \ > @@ -220,11 +220,11 @@ static inline void sched_cacheflush(void > /* > * Enable IRQs > */ > -#define local_irq_enable() \ > +#define local_irq_enable_hw() \ > ({ \ > unsigned long temp; \ > __asm__ __volatile__( \ > - "mrs %0, cpsr @ local_irq_enable\n" \ > + "mrs %0, cpsr @ local_irq_enable_hw\n"\ > " bic %0, %0, #128\n" \ > " msr cpsr_c, %0" \ > : "=r" (temp) \ > @@ -235,11 +235,11 @@ static inline void sched_cacheflush(void > /* > * Disable IRQs > */ > -#define local_irq_disable() \ > +#define local_irq_disable_hw() \ > ({ \ > unsigned long temp; \ > __asm__ __volatile__( \ > - "mrs %0, cpsr @ local_irq_disable\n" \ > + "mrs %0, cpsr @ local_irq_disable_hw\n"\ > " orr %0, %0, #128\n" \ > " msr cpsr_c, %0" \ > : "=r" (temp) \ > @@ -250,7 +250,7 @@ static inline void sched_cacheflush(void > /* > * Enable FIQs > */ > -#define local_fiq_enable() \ > +#define local_fiq_enable_hw() \ > ({ \ > unsigned long temp; \ > __asm__ __volatile__( \ > @@ -265,7 +265,7 @@ static inline void sched_cacheflush(void > /* > * Disable FIQs > */ > -#define local_fiq_disable() \ > +#define local_fiq_disable_hw() \ > ({ \ > unsigned long temp; \ > __asm__ __volatile__( \ > @@ -282,29 +282,63 @@ static inline void sched_cacheflush(void > /* > * Save the current interrupt enable state. > */ > -#define local_save_flags(x) \ > +#define local_save_flags_hw(x) \ > ({ \ > __asm__ __volatile__( \ > - "mrs %0, cpsr @ local_save_flags" \ > + "mrs %0, cpsr @ local_save_flags_hw" \ > : "=r" (x) : : "memory", "cc"); \ > }) > > /* > * restore saved IRQ & FIQ state > */ > -#define local_irq_restore(x) \ > +#define local_irq_restore_hw(x) \ > __asm__ __volatile__( \ > - "msr cpsr_c, %0 @ local_irq_restore\n" \ > + "msr cpsr_c, %0 @ local_irq_restore_hw\n"\ > : \ > : "r" (x) \ > : "memory", "cc") > > -#define irqs_disabled() \ > -({ \ > +#define irqs_disabled_hw() \ > + ({ \ > unsigned long flags; \ > - local_save_flags(flags); \ > + local_save_flags_hw(flags); \ > (int)(flags & PSR_I_BIT); \ > -}) > + }) > + > + > +#ifdef CONFIG_IPIPE > + > +void __ipipe_stall_root(void); > +void __ipipe_unstall_root(void); > +unsigned long __ipipe_test_root(void); > +unsigned long __ipipe_test_and_stall_root(void); > +void __ipipe_restore_root(unsigned long flags); > + > +/* PSR_I_BIT is bit no. 7 and is set if interrupts are _disabled_ */ > +#define local_irq_save(flags) ((flags) = __ipipe_test_and_stall_root() << 7) > +#define local_irq_enable() __ipipe_unstall_root() > +#define local_irq_disable() __ipipe_stall_root() > +#define local_fiq_enable() __ipipe_unstall_root() > +#define local_fiq_disable() __ipipe_stall_root() > +#define local_save_flags(flags) ((flags) = __ipipe_test_root() << 7) > +#define local_irq_restore(flags) __ipipe_restore_root(flags & (1 << 7)) > + > +#define irqs_disabled() __ipipe_test_root() > + > +#else /* !CONFIG_IPIPE */ > + > +#define local_irq_save(flags) local_irq_save_hw(flags) > +#define local_irq_enable() local_irq_enable_hw() > +#define local_irq_disable() local_irq_disable_hw() > +#define local_fiq_enable() local_fiq_enable_hw() > +#define local_fiq_disable() local_fiq_disable_hw() > +#define local_save_flags(flags) local_save_flags_hw(flags) > +#define local_irq_restore(flags) local_irq_restore_hw(flags) > + > +#define irqs_disabled() irqs_disabled_hw() > + > +#endif /* CONFIG_IPIPE */ > > #ifdef CONFIG_SMP > > @@ -379,17 +413,17 @@ static inline unsigned long __xchg(unsig > #error SMP is not supported on this platform > #endif > case 1: > - local_irq_save(flags); > + local_irq_save_hw(flags); > ret = *(volatile unsigned char *)ptr; > *(volatile unsigned char *)ptr = x; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > break; > > case 4: > - local_irq_save(flags); > + local_irq_save_hw(flags); > ret = *(volatile unsigned long *)ptr; > *(volatile unsigned long *)ptr = x; > - local_irq_restore(flags); > + local_irq_restore_hw(flags); > break; > #else > case 1: > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/hardirq.h linux-2.6.16.5-tcl1-ipipe/include/linux/hardirq.h > --- linux-2.6.16.5-tcl1/include/linux/hardirq.h 2006-05-07 16:42:11.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/linux/hardirq.h 2006-07-16 15:01:24.000000000 +0200 > @@ -87,8 +87,21 @@ extern void synchronize_irq(unsigned int > # define synchronize_irq(irq) barrier() > #endif > > +#ifdef CONFIG_IPIPE > +#define nmi_enter() \ > +do { \ > + if (ipipe_current_domain == ipipe_root_domain) \ > + irq_enter(); \ > +} while(0) > +#define nmi_exit() \ > +do { \ > + if (ipipe_current_domain == ipipe_root_domain) \ > + sub_preempt_count(HARDIRQ_OFFSET); \ > +} while(0) > +#else /* !CONFIG_IPIPE */ > #define nmi_enter() irq_enter() > #define nmi_exit() sub_preempt_count(HARDIRQ_OFFSET) > +#endif /* CONFIG_IPIPE */ > > struct task_struct; > > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/linux/ipipe.h > --- linux-2.6.16.5-tcl1/include/linux/ipipe.h 1970-01-01 01:00:00.000000000 +0100 > +++ linux-2.6.16.5-tcl1-ipipe/include/linux/ipipe.h 2006-07-16 15:01:24.000000000 +0200 > @@ -0,0 +1,699 @@ > +/* -*- linux-c -*- > + * include/linux/ipipe.h > + * > + * Copyright (C) 2002-2005 Philippe Gerum. > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, > + * USA; either version 2 of the License, or (at your option) any later > + * version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. > + */ > + > +#ifndef __LINUX_IPIPE_H > +#define __LINUX_IPIPE_H > + > +#include <linux/config.h> > +#include <linux/spinlock.h> > +#include <linux/cache.h> > +#include <asm/ipipe.h> > + > +#ifdef CONFIG_IPIPE > + > +#define IPIPE_VERSION_STRING IPIPE_ARCH_STRING > +#define IPIPE_RELEASE_NUMBER ((IPIPE_MAJOR_NUMBER << 16) | \ > + (IPIPE_MINOR_NUMBER << 8) | \ > + (IPIPE_PATCH_NUMBER)) > + > +#ifndef BROKEN_BUILTIN_RETURN_ADDRESS > +#define __BUILTIN_RETURN_ADDRESS0 ((unsigned long)__builtin_return_address(0)) > +#define __BUILTIN_RETURN_ADDRESS1 ((unsigned long)__builtin_return_address(1)) > +#endif /* !BUILTIN_RETURN_ADDRESS */ > + > +#define IPIPE_ROOT_PRIO 100 > +#define IPIPE_ROOT_ID 0 > +#define IPIPE_ROOT_NPTDKEYS 4 /* Must be <= BITS_PER_LONG */ > + > +#define IPIPE_RESET_TIMER 0x1 > +#define IPIPE_GRAB_TIMER 0x2 > + > +/* Global domain flags */ > +#define IPIPE_SPRINTK_FLAG 0 /* Synchronous printk() allowed */ > +#define IPIPE_AHEAD_FLAG 1 /* Domain always heads the pipeline */ > + > +#define IPIPE_STALL_FLAG 0 /* Stalls a pipeline stage -- guaranteed at bit #0 */ > +#define IPIPE_SYNC_FLAG 1 /* The interrupt syncer is running for the domain */ > + > +#define IPIPE_SYNC_MASK (1 << IPIPE_SYNC_FLAG) > + > +#define IPIPE_HANDLE_FLAG 0 > +#define IPIPE_PASS_FLAG 1 > +#define IPIPE_ENABLE_FLAG 2 > +#define IPIPE_DYNAMIC_FLAG IPIPE_HANDLE_FLAG > +#define IPIPE_STICKY_FLAG 3 > +#define IPIPE_SYSTEM_FLAG 4 > +#define IPIPE_LOCK_FLAG 5 > +#define IPIPE_SHARED_FLAG 6 > +#define IPIPE_WIRED_FLAG 7 > +#define IPIPE_EXCLUSIVE_FLAG 8 > + > +#define IPIPE_HANDLE_MASK (1 << IPIPE_HANDLE_FLAG) > +#define IPIPE_PASS_MASK (1 << IPIPE_PASS_FLAG) > +#define IPIPE_ENABLE_MASK (1 << IPIPE_ENABLE_FLAG) > +#define IPIPE_DYNAMIC_MASK IPIPE_HANDLE_MASK > +#define IPIPE_STICKY_MASK (1 << IPIPE_STICKY_FLAG) > +#define IPIPE_SYSTEM_MASK (1 << IPIPE_SYSTEM_FLAG) > +#define IPIPE_LOCK_MASK (1 << IPIPE_LOCK_FLAG) > +#define IPIPE_SHARED_MASK (1 << IPIPE_SHARED_FLAG) > +#define IPIPE_WIRED_MASK (1 << IPIPE_WIRED_FLAG) > +#define IPIPE_EXCLUSIVE_MASK (1 << IPIPE_EXCLUSIVE_FLAG) > + > +#define IPIPE_DEFAULT_MASK (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK) > +#define IPIPE_STDROOT_MASK (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK|IPIPE_SYSTEM_MASK) > + > +#define IPIPE_EVENT_SELF 0x80000000 > + > +/* Number of virtual IRQs */ > +#define IPIPE_NR_VIRQS BITS_PER_LONG > +/* First virtual IRQ # */ > +#define IPIPE_VIRQ_BASE (((IPIPE_NR_XIRQS + BITS_PER_LONG - 1) / BITS_PER_LONG) * BITS_PER_LONG) > +/* Total number of IRQ slots */ > +#define IPIPE_NR_IRQS (IPIPE_VIRQ_BASE + IPIPE_NR_VIRQS) > +/* Number of indirect words needed to map the whole IRQ space. */ > +#define IPIPE_IRQ_IWORDS ((IPIPE_NR_IRQS + BITS_PER_LONG - 1) / BITS_PER_LONG) > +#define IPIPE_IRQ_IMASK (BITS_PER_LONG - 1) > +#define IPIPE_IRQMASK_ANY (~0L) > +#define IPIPE_IRQMASK_VIRT (IPIPE_IRQMASK_ANY << (IPIPE_VIRQ_BASE / BITS_PER_LONG)) > + > +#ifdef CONFIG_SMP > + > +#define IPIPE_NR_CPUS NR_CPUS > +#define ipipe_declare_cpuid int cpuid > +#define ipipe_load_cpuid() do { \ > + (cpuid) = ipipe_processor_id(); \ > + } while(0) > +#define ipipe_lock_cpu(flags) do { \ > + local_irq_save_hw(flags); \ > + (cpuid) = ipipe_processor_id(); \ > + } while(0) > +#define ipipe_unlock_cpu(flags) local_irq_restore_hw(flags) > +#define ipipe_get_cpu(flags) ipipe_lock_cpu(flags) > +#define ipipe_put_cpu(flags) ipipe_unlock_cpu(flags) > +#define ipipe_current_domain (ipipe_percpu_domain[ipipe_processor_id()]) > + > +#else /* !CONFIG_SMP */ > + > +#define IPIPE_NR_CPUS 1 > +#define ipipe_declare_cpuid const int cpuid = 0 > +#define ipipe_load_cpuid() do { } while(0) > +#define ipipe_lock_cpu(flags) local_irq_save_hw(flags) > +#define ipipe_unlock_cpu(flags) local_irq_restore_hw(flags) > +#define ipipe_get_cpu(flags) do { (void)(flags); } while(0) > +#define ipipe_put_cpu(flags) do { } while(0) > +#define ipipe_current_domain (ipipe_percpu_domain[0]) > + > +#endif /* CONFIG_SMP */ > + > +#define ipipe_virtual_irq_p(irq) ((irq) >= IPIPE_VIRQ_BASE && \ > + (irq) < IPIPE_NR_IRQS) > + > +typedef void (*ipipe_irq_handler_t)(unsigned irq, > + void *cookie); > + > +#define IPIPE_SAME_HANDLER ((ipipe_irq_handler_t)(-1)) > + > +typedef int (*ipipe_irq_ackfn_t)(unsigned irq); > + > +typedef int (*ipipe_event_handler_t)(unsigned event, > + struct ipipe_domain *from, > + void *data); > +struct ipipe_domain { > + > + struct list_head p_link; /* Link in pipeline */ > + > + struct ipcpudata { > + unsigned long status; > + unsigned long irq_pending_hi; > + unsigned long irq_pending_lo[IPIPE_IRQ_IWORDS]; > + struct ipirqcnt { > + unsigned long pending_hits; > + unsigned long total_hits; > + } irq_counters[IPIPE_NR_IRQS]; > + } ____cacheline_aligned_in_smp cpudata[IPIPE_NR_CPUS]; > + > + struct { > + unsigned long control; > + ipipe_irq_ackfn_t acknowledge; > + ipipe_irq_handler_t handler; > + void *cookie; > + } ____cacheline_aligned irqs[IPIPE_NR_IRQS]; > + > + ipipe_event_handler_t evhand[IPIPE_NR_EVENTS]; /* Event handlers. */ > + unsigned long long evself; /* Self-monitored event bits. */ > + unsigned long flags; > + unsigned domid; > + const char *name; > + int priority; > + void *pdd; > +}; > + > +#define IPIPE_HEAD_PRIORITY (-1) /* For domains always heading the pipeline */ > + > +struct ipipe_domain_attr { > + > + unsigned domid; /* Domain identifier -- Magic value set by caller */ > + const char *name; /* Domain name -- Warning: won't be dup'ed! */ > + int priority; /* Priority in interrupt pipeline */ > + void (*entry) (void); /* Domain entry point */ > + void *pdd; /* Per-domain (opaque) data pointer */ > +}; > + > +/* The following macros must be used hw interrupts off. */ > + > +#define __ipipe_irq_cookie(ipd,irq) (ipd)->irqs[irq].cookie > +#define __ipipe_irq_handler(ipd,irq) (ipd)->irqs[irq].handler > + > +#define __ipipe_cpudata_irq_hits(ipd,cpuid,irq) ((ipd)->cpudata[cpuid].irq_counters[irq].total_hits) > + > +#define __ipipe_set_irq_bit(ipd,cpuid,irq) \ > +do { \ > + if (!test_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) { \ > + __set_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ > + __set_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[cpuid].irq_pending_hi); \ > + } \ > +} while(0) > + > +#define __ipipe_clear_pend(ipd,cpuid,irq) \ > +do { \ > + __clear_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ > + if ((ipd)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT] == 0) \ > + __clear_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[cpuid].irq_pending_hi); \ > +} while(0) > + > +#define __ipipe_lock_irq(ipd,cpuid,irq) \ > +do { \ > + if (!test_and_set_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) \ > + __ipipe_clear_pend(ipd,cpuid,irq); \ > +} while(0) > + > +#define __ipipe_unlock_irq(ipd,irq) \ > +do { \ > + int __cpuid, __nr_cpus = num_online_cpus(); \ > + if (test_and_clear_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control)) \ > + for (__cpuid = 0; __cpuid < __nr_cpus; __cpuid++) \ > + if ((ipd)->cpudata[__cpuid].irq_counters[irq].pending_hits > 0) { /* We need atomic ops next. */ \ > + set_bit(irq & IPIPE_IRQ_IMASK,&(ipd)->cpudata[__cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ > + set_bit(irq >> IPIPE_IRQ_ISHIFT,&(ipd)->cpudata[__cpuid].irq_pending_hi); \ > + } \ > +} while(0) > + > +#define __ipipe_clear_irq(ipd,irq) \ > +do { \ > + int __cpuid, __nr_cpus = num_online_cpus(); \ > + clear_bit(IPIPE_LOCK_FLAG,&(ipd)->irqs[irq].control); \ > + for (__cpuid = 0; __cpuid < __nr_cpus; __cpuid++) { \ > + (ipd)->cpudata[__cpuid].irq_counters[irq].pending_hits = 0; \ > + __ipipe_clear_pend(ipd,__cpuid,irq); \ > + } \ > +} while(0) > + > +#ifdef __RAW_SPIN_LOCK_UNLOCKED > +#define spin_lock_hw(x) _raw_spin_lock(x) > +#define spin_trylock_hw(x) _raw_spin_trylock(x) > +#define spin_unlock_hw(x) _raw_spin_unlock(x) > +#if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK) > +#define write_lock_hw(x) _raw_write_lock(x) > +#define write_trylock_hw(x) _raw_write_trylock(x) > +#define write_unlock_hw(x) _raw_write_unlock(x) > +#define read_lock_hw(x) _raw_read_lock(x) > +#define read_trylock_hw(x) _raw_read_trylock(x) > +#define read_unlock_hw(x) _raw_read_unlock(x) > +#else /* UP non-debug */ > +#define write_lock_hw(lock) do { (void)(lock); } while (0) > +#define write_trylock_hw(lock) ({ (void)(lock); 1; }) > +#define write_unlock_hw(lock) do { (void)(lock); } while (0) > +#define read_lock_hw(lock) do { (void)(lock); } while (0) > +#define read_trylock_hw(lock) ({ (void)(lock); 1; }) > +#define read_unlock_hw(lock) do { (void)(lock); } while (0) > +#endif /* CONFIG_SMP || CONFIG_DEBUG_SPINLOCK */ > +#else /* !__RAW_SPIN_LOCK_UNLOCKED */ > +#define spin_lock_hw(x) _spin_lock(x) > +#define spin_unlock_hw(x) _spin_unlock(x) > +#define spin_trylock_hw(x) _spin_trylock(x) > +#define write_lock_hw(x) _write_lock(x) > +#define write_unlock_hw(x) _write_unlock(x) > +#define write_trylock_hw(x) _write_trylock(x) > +#define read_lock_hw(x) _read_lock(x) > +#define read_unlock_hw(x) _read_unlock(x) > +#endif /* __RAW_SPIN_LOCK_UNLOCKED */ > + > +typedef spinlock_t ipipe_spinlock_t; > +typedef rwlock_t ipipe_rwlock_t; > +#define IPIPE_SPIN_LOCK_UNLOCKED SPIN_LOCK_UNLOCKED > +#define IPIPE_RW_LOCK_UNLOCKED RW_LOCK_UNLOCKED > + > +#define spin_lock_irqsave_hw(x,flags) \ > +do { \ > + local_irq_save_hw(flags); \ > + spin_lock_hw(x); \ > +} while (0) > + > +#define spin_unlock_irqrestore_hw(x,flags) \ > +do { \ > + spin_unlock_hw(x); \ > + local_irq_restore_hw(flags); \ > +} while (0) > + > +#define spin_lock_irq_hw(x) \ > +do { \ > + local_irq_disable_hw(); \ > + spin_lock_hw(x); \ > +} while (0) > + > +#define spin_unlock_irq_hw(x) \ > +do { \ > + spin_unlock_hw(x); \ > + local_irq_enable_hw(); \ > +} while (0) > + > +#define read_lock_irqsave_hw(lock, flags) \ > +do { \ > + local_irq_save_hw(flags); \ > + read_lock_hw(lock); \ > +} while (0) > + > +#define read_unlock_irqrestore_hw(lock, flags) \ > +do { \ > + read_unlock_hw(lock); \ > + local_irq_restore_hw(flags); \ > +} while (0) > + > +#define write_lock_irqsave_hw(lock, flags) \ > +do { \ > + local_irq_save_hw(flags); \ > + write_lock_hw(lock); \ > +} while (0) > + > +#define write_unlock_irqrestore_hw(lock, flags) \ > +do { \ > + write_unlock_hw(lock); \ > + local_irq_restore_hw(flags); \ > +} while (0) > + > +extern struct ipipe_domain *ipipe_percpu_domain[], *ipipe_root_domain; > + > +extern unsigned __ipipe_printk_virq; > + > +extern unsigned long __ipipe_virtual_irq_map; > + > +extern struct list_head __ipipe_pipeline; > + > +extern ipipe_spinlock_t __ipipe_pipelock; > + > +extern int __ipipe_event_monitors[]; > + > +/* Private interface */ > + > +void ipipe_init(void); > + > +#ifdef CONFIG_PROC_FS > +void ipipe_init_proc(void); > + > +#ifdef CONFIG_IPIPE_TRACE > +void __ipipe_init_trace_proc(void); > +#else /* !CONFIG_IPIPE_TRACE */ > +#define __ipipe_init_trace_proc() do { } while(0) > +#endif /* CONFIG_IPIPE_TRACE */ > + > +#else /* !CONFIG_PROC_FS */ > +#define ipipe_init_proc() do { } while(0) > +#endif /* CONFIG_PROC_FS */ > + > +void __ipipe_init_stage(struct ipipe_domain *ipd); > + > +void __ipipe_cleanup_domain(struct ipipe_domain *ipd); > + > +void __ipipe_add_domain_proc(struct ipipe_domain *ipd); > + > +void __ipipe_remove_domain_proc(struct ipipe_domain *ipd); > + > +void __ipipe_flush_printk(unsigned irq, void *cookie); > + > +void __ipipe_stall_root(void); > + > +void __ipipe_unstall_root(void); > + > +unsigned long __ipipe_test_root(void); > + > +unsigned long __ipipe_test_and_stall_root(void); > + > +void fastcall __ipipe_walk_pipeline(struct list_head *pos, int cpuid); > + > +void fastcall __ipipe_restore_root(unsigned long x); > + > +int fastcall __ipipe_schedule_irq(unsigned irq, struct list_head *head); > + > +int fastcall __ipipe_dispatch_event(unsigned event, void *data); > + > +int fastcall __ipipe_dispatch_wired(struct ipipe_domain *head, unsigned irq); > + > +void fastcall __ipipe_sync_stage(unsigned long syncmask); > + > +#ifndef __ipipe_sync_pipeline > +#define __ipipe_sync_pipeline(syncmask) __ipipe_sync_stage(syncmask) > +#endif > + > +#ifndef __ipipe_run_irqtail > +#define __ipipe_run_irqtail() do { } while(0) > +#endif > + > +#define __ipipe_pipeline_head_p(ipd) (&(ipd)->p_link == __ipipe_pipeline.next) > + > +/* > + * Keep the following as a macro, so that client code could check for > + * the support of the invariant pipeline head optimization. > + */ > +#define __ipipe_pipeline_head() list_entry(__ipipe_pipeline.next,struct ipipe_domain,p_link) > + > +#define __ipipe_event_pipelined_p(ev) \ > + (__ipipe_event_monitors[ev] > 0 || (ipipe_current_domain->evself & (1LL << ev))) > + > +#ifdef CONFIG_SMP > + > +cpumask_t __ipipe_set_irq_affinity(unsigned irq, > + cpumask_t cpumask); > + > +int fastcall __ipipe_send_ipi(unsigned ipi, > + cpumask_t cpumask); > + > +#endif /* CONFIG_SMP */ > + > +/* Called with hw interrupts off. */ > +static inline void __ipipe_switch_to(struct ipipe_domain *out, > + struct ipipe_domain *in, int cpuid) > +{ > + void ipipe_suspend_domain(void); > + > + /* > + * "in" is guaranteed to be closer than "out" from the head of the > + * pipeline (and obviously different). > + */ > + > + ipipe_percpu_domain[cpuid] = in; > + > + ipipe_suspend_domain(); /* Sync stage and propagate interrupts. */ > + ipipe_load_cpuid(); /* Processor might have changed. */ > + > + if (ipipe_percpu_domain[cpuid] == in) > + /* > + * Otherwise, something has changed the current domain under > + * our feet recycling the register set; do not override. > + */ > + ipipe_percpu_domain[cpuid] = out; > +} > + > +static inline void ipipe_sigwake_notify(struct task_struct *p) > +{ > + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SIGWAKE)) > + __ipipe_dispatch_event(IPIPE_EVENT_SIGWAKE,p); > +} > + > +static inline void ipipe_setsched_notify(struct task_struct *p) > +{ > + if (__ipipe_event_pipelined_p(IPIPE_EVENT_SETSCHED)) > + __ipipe_dispatch_event(IPIPE_EVENT_SETSCHED,p); > +} > + > +static inline void ipipe_exit_notify(struct task_struct *p) > +{ > + if (__ipipe_event_pipelined_p(IPIPE_EVENT_EXIT)) > + __ipipe_dispatch_event(IPIPE_EVENT_EXIT,p); > +} > + > +static inline int ipipe_trap_notify(int ex, struct pt_regs *regs) > +{ > + return __ipipe_event_pipelined_p(ex) ? __ipipe_dispatch_event(ex,regs) : 0; > +} > + > +struct mm_struct; > + > +static inline void ipipe_cleanup_notify(struct mm_struct *mm) > +{ > + if (__ipipe_event_pipelined_p(IPIPE_EVENT_CLEANUP)) > + __ipipe_dispatch_event(IPIPE_EVENT_CLEANUP,mm); > +} > + > +/* Public interface */ > + > +int ipipe_register_domain(struct ipipe_domain *ipd, > + struct ipipe_domain_attr *attr); > + > +int ipipe_unregister_domain(struct ipipe_domain *ipd); > + > +void ipipe_suspend_domain(void); > + > +int ipipe_virtualize_irq(struct ipipe_domain *ipd, > + unsigned irq, > + ipipe_irq_handler_t handler, > + void *cookie, > + ipipe_irq_ackfn_t acknowledge, > + unsigned modemask); > + > +static inline int ipipe_share_irq(unsigned irq, > + ipipe_irq_ackfn_t acknowledge) > +{ > + return ipipe_virtualize_irq(ipipe_current_domain, > + irq, > + IPIPE_SAME_HANDLER, > + NULL, > + acknowledge, > + IPIPE_SHARED_MASK | IPIPE_HANDLE_MASK | > + IPIPE_PASS_MASK); > +} > + > +int ipipe_control_irq(unsigned irq, > + unsigned clrmask, > + unsigned setmask); > + > +unsigned ipipe_alloc_virq(void); > + > +int ipipe_free_virq(unsigned virq); > + > +int fastcall ipipe_trigger_irq(unsigned irq); > + > +static inline int ipipe_propagate_irq(unsigned irq) > +{ > + return __ipipe_schedule_irq(irq, ipipe_current_domain->p_link.next); > +} > + > +static inline int ipipe_schedule_irq(unsigned irq) > +{ > + return __ipipe_schedule_irq(irq, &ipipe_current_domain->p_link); > +} > + > +void fastcall ipipe_stall_pipeline_from(struct ipipe_domain *ipd); > + > +unsigned long fastcall ipipe_test_and_stall_pipeline_from(struct ipipe_domain *ipd); > + > +void fastcall ipipe_unstall_pipeline_from(struct ipipe_domain *ipd); > + > +unsigned long fastcall ipipe_test_and_unstall_pipeline_from(struct ipipe_domain *ipd); > + > +void fastcall ipipe_restore_pipeline_from(struct ipipe_domain *ipd, > + unsigned long x); > + > +static inline unsigned long ipipe_test_pipeline_from(struct ipipe_domain *ipd) > +{ > + unsigned long flags, x; > + ipipe_declare_cpuid; > + > + ipipe_get_cpu(flags); > + x = test_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > + ipipe_put_cpu(flags); > + > + return x; > +} > + > +static inline void ipipe_restore_pipeline_nosync(struct ipipe_domain *ipd, > + unsigned long x, int cpuid) > +{ > + /* > + * If cpuid is current, then it must be held on entry > + * (ipipe_get_cpu/local_irq_save_hw/local_irq_disable_hw). > + */ > + > + if (x) > + __set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > + else > + __clear_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > +} > + > +static inline void ipipe_stall_pipeline_head(void) > +{ > + ipipe_declare_cpuid; > + unsigned long flags; > + > + ipipe_lock_cpu(flags); > + __set_bit(IPIPE_STALL_FLAG, &__ipipe_pipeline_head()->cpudata[cpuid].status); > +} > + > +static inline unsigned long ipipe_test_and_stall_pipeline_head(void) > +{ > + unsigned long flags; > + ipipe_declare_cpuid; > + > + ipipe_lock_cpu(flags); > + return __test_and_set_bit(IPIPE_STALL_FLAG, &__ipipe_pipeline_head()->cpudata[cpuid].status); > +} > + > +void ipipe_unstall_pipeline_head(void); > + > +void fastcall __ipipe_restore_pipeline_head(struct ipipe_domain *head, > + unsigned long x); > + > +static inline void ipipe_restore_pipeline_head(unsigned long x) > +{ > + struct ipipe_domain *head = __ipipe_pipeline_head(); > + /* On some archs, __test_and_set_bit() might return different > + * truth value than test_bit(), so we test the exclusive OR of > + * both statuses, assuming that the lowest bit is always set in > + * the truth value (if this is wrong, the failed optimization will > + * be caught in __ipipe_restore_pipeline_head() if > + * CONFIG_DEBUG_KERNEL is set). */ > + if ((x ^ test_bit(IPIPE_STALL_FLAG, &head->cpudata[ipipe_processor_id()].status)) & 1) > + __ipipe_restore_pipeline_head(head,x); > +} > + > +#define ipipe_unstall_pipeline() \ > + ipipe_unstall_pipeline_from(ipipe_current_domain) > + > +#define ipipe_test_and_unstall_pipeline() \ > + ipipe_test_and_unstall_pipeline_from(ipipe_current_domain) > + > +#define ipipe_test_pipeline() \ > + ipipe_test_pipeline_from(ipipe_current_domain) > + > +#define ipipe_test_and_stall_pipeline() \ > + ipipe_test_and_stall_pipeline_from(ipipe_current_domain) > + > +#define ipipe_stall_pipeline() \ > + ipipe_stall_pipeline_from(ipipe_current_domain) > + > +#define ipipe_restore_pipeline(x) \ > + ipipe_restore_pipeline_from(ipipe_current_domain, (x)) > + > +void ipipe_init_attr(struct ipipe_domain_attr *attr); > + > +int ipipe_get_sysinfo(struct ipipe_sysinfo *sysinfo); > + > +int ipipe_tune_timer(unsigned long ns, > + int flags); > + > +unsigned long ipipe_critical_enter(void (*syncfn) (void)); > + > +void ipipe_critical_exit(unsigned long flags); > + > +static inline void ipipe_set_printk_sync(struct ipipe_domain *ipd) > +{ > + set_bit(IPIPE_SPRINTK_FLAG, &ipd->flags); > +} > + > +static inline void ipipe_set_printk_async(struct ipipe_domain *ipd) > +{ > + clear_bit(IPIPE_SPRINTK_FLAG, &ipd->flags); > +} > + > +ipipe_event_handler_t ipipe_catch_event(struct ipipe_domain *ipd, > + unsigned event, > + ipipe_event_handler_t handler); > + > +cpumask_t ipipe_set_irq_affinity(unsigned irq, > + cpumask_t cpumask); > + > +int fastcall ipipe_send_ipi(unsigned ipi, > + cpumask_t cpumask); > + > +int ipipe_setscheduler_root(struct task_struct *p, > + int policy, > + int prio); > + > +int ipipe_reenter_root(struct task_struct *prev, > + int policy, > + int prio); > + > +int ipipe_alloc_ptdkey(void); > + > +int ipipe_free_ptdkey(int key); > + > +int fastcall ipipe_set_ptd(int key, > + void *value); > + > +void fastcall *ipipe_get_ptd(int key); > + > +#define local_irq_enable_hw_cond() local_irq_enable_hw() > +#define local_irq_disable_hw_cond() local_irq_disable_hw() > +#define local_irq_save_hw_cond(flags) local_irq_save_hw(flags) > +#define local_irq_restore_hw_cond(flags) local_irq_restore_hw(flags) > +#define spin_lock_irqsave_hw_cond(lock,flags) spin_lock_irqsave_hw(lock,flags) > +#define spin_unlock_irqrestore_hw_cond(lock,flags) spin_unlock_irqrestore_hw(lock,flags) > +#define smp_processor_id_hw() ipipe_processor_id() > + > +#define ipipe_irq_lock(irq) \ > + do { \ > + ipipe_declare_cpuid; \ > + ipipe_load_cpuid(); \ > + __ipipe_lock_irq(ipipe_percpu_domain[cpuid], cpuid, irq);\ > + } while(0) > + > +#define ipipe_irq_unlock(irq) \ > + do { \ > + ipipe_declare_cpuid; \ > + ipipe_load_cpuid(); \ > + __ipipe_unlock_irq(ipipe_percpu_domain[cpuid], irq); \ > + } while(0) > + > +#define ipipe_root_domain_p (ipipe_current_domain == ipipe_root_domain) > + > +#else /* !CONFIG_IPIPE */ > + > +#define ipipe_init() do { } while(0) > +#define ipipe_suspend_domain() do { } while(0) > +#define ipipe_sigwake_notify(p) do { } while(0) > +#define ipipe_setsched_notify(p) do { } while(0) > +#define ipipe_exit_notify(p) do { } while(0) > +#define ipipe_init_proc() do { } while(0) > +#define ipipe_trap_notify(t,r) 0 > +#define ipipe_cleanup_notify(mm) do { } while(0) > + > +#define spin_lock_hw(lock) spin_lock(lock) > +#define spin_unlock_hw(lock) spin_unlock(lock) > +#define spin_lock_irq_hw(lock) spin_lock_irq(lock) > +#define spin_unlock_irq_hw(lock) spin_unlock_irq(lock) > +#define spin_lock_irqsave_hw(lock,flags) spin_lock_irqsave(lock, flags) > +#define spin_unlock_irqrestore_hw(lock,flags) spin_unlock_irqrestore(lock, flags) > + > +#define local_irq_enable_hw_cond() do { } while(0) > +#define local_irq_disable_hw_cond() do { } while(0) > +#define local_irq_save_hw_cond(flags) do { (void)(flags); } while(0) > +#define local_irq_restore_hw_cond(flags) do { } while(0) > +#define spin_lock_irqsave_hw_cond(lock,flags) do { (void)(flags); spin_lock(lock); } while(0) > +#define spin_unlock_irqrestore_hw_cond(lock,flags) spin_unlock(lock) > +#define smp_processor_id_hw() smp_processor_id() > + > +#define ipipe_irq_lock(irq) do { } while(0) > +#define ipipe_irq_unlock(irq) do { } while(0) > + > +#define ipipe_root_domain_p 1 > + > +#endif /* CONFIG_IPIPE */ > + > +#endif /* !__LINUX_IPIPE_H */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/linkage.h linux-2.6.16.5-tcl1-ipipe/include/linux/linkage.h > --- linux-2.6.16.5-tcl1/include/linux/linkage.h 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/linux/linkage.h 2006-07-16 15:01:24.000000000 +0200 > @@ -51,4 +51,8 @@ > #define fastcall > #endif > > +#ifndef notrace > +#define notrace __attribute__((no_instrument_function)) > +#endif > + > #endif > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/preempt.h linux-2.6.16.5-tcl1-ipipe/include/linux/preempt.h > --- linux-2.6.16.5-tcl1/include/linux/preempt.h 2006-05-07 15:37:01.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/linux/preempt.h 2006-07-16 15:01:24.000000000 +0200 > @@ -27,29 +27,43 @@ > > asmlinkage void preempt_schedule(void); > > -#define preempt_disable() \ > -do { \ > - inc_preempt_count(); \ > - barrier(); \ > +#ifdef CONFIG_IPIPE > +#include <asm/ipipe.h> > +extern struct ipipe_domain *ipipe_percpu_domain[], *ipipe_root_domain; > +#define ipipe_preempt_guard() (ipipe_percpu_domain[ipipe_processor_id()] == ipipe_root_domain) > +#else /* !CONFIG_IPIPE */ > +#define ipipe_preempt_guard() 1 > +#endif /* CONFIG_IPIPE */ > + > +#define preempt_disable() \ > +do { \ > + if (ipipe_preempt_guard()) { \ > + inc_preempt_count(); \ > + barrier(); \ > + } \ > } while (0) > > -#define preempt_enable_no_resched() \ > -do { \ > - barrier(); \ > - dec_preempt_count(); \ > +#define preempt_enable_no_resched() \ > +do { \ > + if (ipipe_preempt_guard()) { \ > + barrier(); \ > + dec_preempt_count(); \ > + } \ > } while (0) > > -#define preempt_check_resched() \ > -do { \ > - if (unlikely(test_thread_flag(TIF_NEED_RESCHED))) \ > - preempt_schedule(); \ > +#define preempt_check_resched() \ > +do { \ > + if (ipipe_preempt_guard()) { \ > + if (unlikely(test_thread_flag(TIF_NEED_RESCHED))) \ > + preempt_schedule(); \ > + } \ > } while (0) > > -#define preempt_enable() \ > -do { \ > - preempt_enable_no_resched(); \ > +#define preempt_enable() \ > +do { \ > + preempt_enable_no_resched(); \ > barrier(); \ > - preempt_check_resched(); \ > + preempt_check_resched(); \ > } while (0) > > #else > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/include/linux/sched.h linux-2.6.16.5-tcl1-ipipe/include/linux/sched.h > --- linux-2.6.16.5-tcl1/include/linux/sched.h 2006-05-07 16:42:13.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/linux/sched.h 2006-07-16 15:06:26.000000000 +0200 > @@ -4,6 +4,7 @@ > #include <asm/param.h> /* for HZ */ > > #include <linux/config.h> > +#include <linux/ipipe.h> > #include <linux/capability.h> > #include <linux/threads.h> > #include <linux/kernel.h> > @@ -129,6 +130,11 @@ extern unsigned long nr_iowait(void); > #define EXIT_DEAD 32 > /* in tsk->state again */ > #define TASK_NONINTERACTIVE 64 > +#ifdef CONFIG_IPIPE > +#define TASK_ATOMICSWITCH 512 > +#else /* !CONFIG_IPIPE */ > +#define TASK_ATOMICSWITCH 0 > +#endif /* CONFIG_IPIPE */ > > #define __set_task_state(tsk, state_value) \ > do { (tsk)->state = (state_value); } while (0) > @@ -871,6 +877,9 @@ struct task_struct { > #endif > atomic_t fs_excl; /* holding fs exclusive resources */ > struct rcu_head rcu; > +#ifdef CONFIG_IPIPE > + void *ptd[IPIPE_ROOT_NPTDKEYS]; > +#endif > }; > > static inline pid_t process_group(struct task_struct *tsk) > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/init/Kconfig linux-2.6.16.5-tcl1-ipipe/init/Kconfig > --- linux-2.6.16.5-tcl1/init/Kconfig 2006-05-07 16:42:14.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/init/Kconfig 2006-07-16 15:01:24.000000000 +0200 > @@ -58,6 +58,7 @@ menu "General setup" > > config LOCALVERSION > string "Local version - append to kernel release" > + default "-ipipe" > help > Append an extra string to the end of your kernel version. > This will show up when you type uname, for example. > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/init/main.c linux-2.6.16.5-tcl1-ipipe/init/main.c > --- linux-2.6.16.5-tcl1/init/main.c 2006-07-15 20:06:03.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/init/main.c 2006-07-16 15:01:24.000000000 +0200 > @@ -488,6 +488,11 @@ asmlinkage void __init start_kernel(void > hrtimers_init(); > softirq_init(); > time_init(); > + /* > + * We need to wait for the interrupt and time subsystems to be > + * initialized before enabling the pipeline. > + */ > + ipipe_init(); > > /* > * HACK ALERT! This is early. We're enabling the console before > @@ -613,6 +618,7 @@ static void __init do_basic_setup(void) > #ifdef CONFIG_SYSCTL > sysctl_init(); > #endif > + ipipe_init_proc(); > > do_initcalls(); > } > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/Makefile linux-2.6.16.5-tcl1-ipipe/kernel/Makefile > --- linux-2.6.16.5-tcl1/kernel/Makefile 2006-05-07 16:42:15.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/Makefile 2006-07-16 15:01:24.000000000 +0200 > @@ -34,6 +34,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl > obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ > obj-$(CONFIG_SECCOMP) += seccomp.o > obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o > +obj-$(CONFIG_IPIPE) += ipipe/ > > ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y) > # According to Alan Modra <alan@domain.hid>, the -fno-omit-frame-pointer is > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/exit.c linux-2.6.16.5-tcl1-ipipe/kernel/exit.c > --- linux-2.6.16.5-tcl1/kernel/exit.c 2006-05-07 16:42:15.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/exit.c 2006-07-16 15:01:24.000000000 +0200 > @@ -852,6 +852,7 @@ fastcall NORET_TYPE void do_exit(long co > exit_itimers(tsk->signal); > acct_process(code); > } > + ipipe_exit_notify(tsk); > exit_mm(tsk); > > exit_sem(tsk); > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/fork.c linux-2.6.16.5-tcl1-ipipe/kernel/fork.c > --- linux-2.6.16.5-tcl1/kernel/fork.c 2006-07-13 15:49:59.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/fork.c 2006-07-16 15:15:30.000000000 +0200 > @@ -371,6 +371,7 @@ void fastcall __mmdrop(struct mm_struct > void mmput(struct mm_struct *mm) > { > if (atomic_dec_and_test(&mm->mm_users)) { > + ipipe_cleanup_notify(mm); > exit_aio(mm); > exit_mmap(mm); > if (!list_empty(&mm->mmlist)) { > @@ -1198,6 +1199,14 @@ static task_t *copy_process(unsigned lon > spin_unlock(¤t->sighand->siglock); > write_unlock_irq(&tasklist_lock); > proc_fork_connector(p); > +#ifdef CONFIG_IPIPE > + { > + int k; > + > + for (k = 0; k < IPIPE_ROOT_NPTDKEYS; k++) > + p->ptd[k] = NULL; > + } > +#endif /* CONFIG_IPIPE */ > return p; > > bad_fork_cleanup_namespace: > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/Kconfig linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Kconfig > --- linux-2.6.16.5-tcl1/kernel/ipipe/Kconfig 1970-01-01 01:00:00.000000000 +0100 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Kconfig 2006-07-16 15:01:24.000000000 +0200 > @@ -0,0 +1,6 @@ > +config IPIPE > + bool "Interrupt pipeline" > + default y > + ---help--- > + Activate this option if you want the interrupt pipeline to be > + compiled in. > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/Makefile linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Makefile > --- linux-2.6.16.5-tcl1/kernel/ipipe/Makefile 1970-01-01 01:00:00.000000000 +0100 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/Makefile 2006-07-16 15:01:24.000000000 +0200 > @@ -0,0 +1,3 @@ > + > +obj-$(CONFIG_IPIPE) += core.o generic.o > +obj-$(CONFIG_IPIPE_TRACE) += tracer.o > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/core.c linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/core.c > --- linux-2.6.16.5-tcl1/kernel/ipipe/core.c 1970-01-01 01:00:00.000000000 +0100 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/core.c 2006-07-16 15:01:24.000000000 +0200 > @@ -0,0 +1,1044 @@ > +/* -*- linux-c -*- > + * linux/kernel/ipipe/core.c > + * > + * Copyright (C) 2002-2005 Philippe Gerum. > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, > + * USA; either version 2 of the License, or (at your option) any later > + * version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. > + * > + * Architecture-independent I-PIPE core support. > + */ > + > +#include <linux/sched.h> > +#include <linux/module.h> > +#include <linux/kallsyms.h> > +#include <linux/interrupt.h> > +#ifdef CONFIG_PROC_FS > +#include <linux/proc_fs.h> > +#endif /* CONFIG_PROC_FS */ > + > +static struct ipipe_domain ipipe_root = > + { .cpudata = {[0 ... IPIPE_NR_CPUS-1] = > + { .status = (1<<IPIPE_STALL_FLAG) } } }; > + > +struct ipipe_domain *ipipe_root_domain = &ipipe_root; > + > +struct ipipe_domain *ipipe_percpu_domain[IPIPE_NR_CPUS] = > + {[0 ... IPIPE_NR_CPUS - 1] = &ipipe_root }; > + > +ipipe_spinlock_t __ipipe_pipelock = IPIPE_SPIN_LOCK_UNLOCKED; > + > +LIST_HEAD(__ipipe_pipeline); > + > +unsigned long __ipipe_virtual_irq_map = 0; > + > +#ifdef CONFIG_PRINTK > +unsigned __ipipe_printk_virq; > +#endif /* CONFIG_PRINTK */ > + > +int __ipipe_event_monitors[IPIPE_NR_EVENTS]; > + > +/* > + * ipipe_init() -- Initialization routine of the IPIPE layer. Called > + * by the host kernel early during the boot procedure. > + */ > +void ipipe_init(void) > +{ > + struct ipipe_domain *ipd = &ipipe_root; > + > + __ipipe_check_platform(); /* Do platform dependent checks first. */ > + > + /* > + * A lightweight registration code for the root domain. We are > + * running on the boot CPU, hw interrupts are off, and > + * secondary CPUs are still lost in space. > + */ > + > + ipd->name = "Linux"; > + ipd->domid = IPIPE_ROOT_ID; > + ipd->priority = IPIPE_ROOT_PRIO; > + > + __ipipe_init_stage(ipd); > + > + INIT_LIST_HEAD(&ipd->p_link); > + list_add_tail(&ipd->p_link, &__ipipe_pipeline); > + > + __ipipe_init_platform(); > + > +#ifdef CONFIG_PRINTK > + __ipipe_printk_virq = ipipe_alloc_virq(); /* Cannot fail here. */ > + ipd->irqs[__ipipe_printk_virq].handler = &__ipipe_flush_printk; > + ipd->irqs[__ipipe_printk_virq].cookie = NULL; > + ipd->irqs[__ipipe_printk_virq].acknowledge = NULL; > + ipd->irqs[__ipipe_printk_virq].control = IPIPE_HANDLE_MASK; > +#endif /* CONFIG_PRINTK */ > + > + __ipipe_enable_pipeline(); > + > + printk(KERN_INFO "I-pipe %s: pipeline enabled.\n", > + IPIPE_VERSION_STRING); > +} > + > +void __ipipe_init_stage(struct ipipe_domain *ipd) > +{ > + int cpuid, n; > + > + for (cpuid = 0; cpuid < IPIPE_NR_CPUS; cpuid++) { > + ipd->cpudata[cpuid].irq_pending_hi = 0; > + > + for (n = 0; n < IPIPE_IRQ_IWORDS; n++) > + ipd->cpudata[cpuid].irq_pending_lo[n] = 0; > + > + for (n = 0; n < IPIPE_NR_IRQS; n++) { > + ipd->cpudata[cpuid].irq_counters[n].total_hits = 0; > + ipd->cpudata[cpuid].irq_counters[n].pending_hits = 0; > + } > + } > + > + for (n = 0; n < IPIPE_NR_IRQS; n++) { > + ipd->irqs[n].acknowledge = NULL; > + ipd->irqs[n].handler = NULL; > + ipd->irqs[n].control = IPIPE_PASS_MASK; /* Pass but don't handle */ > + } > + > + for (n = 0; n < IPIPE_NR_EVENTS; n++) > + ipd->evhand[n] = NULL; > + > + ipd->evself = 0; > + > +#ifdef CONFIG_SMP > + ipd->irqs[IPIPE_CRITICAL_IPI].acknowledge = &__ipipe_ack_system_irq; > + ipd->irqs[IPIPE_CRITICAL_IPI].handler = &__ipipe_do_critical_sync; > + ipd->irqs[IPIPE_CRITICAL_IPI].cookie = NULL; > + /* Immediately handle in the current domain but *never* pass */ > + ipd->irqs[IPIPE_CRITICAL_IPI].control = > + IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK|IPIPE_SYSTEM_MASK; > +#endif /* CONFIG_SMP */ > +} > + > +void __ipipe_stall_root(void) > +{ > + ipipe_declare_cpuid; > + unsigned long flags; > + > + ipipe_get_cpu(flags); /* Care for migration. */ > + > + set_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); > + > +#ifdef CONFIG_SMP > + if (!__ipipe_pipeline_head_p(ipipe_root_domain)) > + ipipe_put_cpu(flags); > +#else /* CONFIG_SMP */ > + if (__ipipe_pipeline_head_p(ipipe_root_domain)) > + local_irq_disable_hw(); > +#endif /* CONFIG_SMP */ > +} > + > +void __ipipe_cleanup_domain(struct ipipe_domain *ipd) > +{ > + ipipe_unstall_pipeline_from(ipd); > + > +#ifdef CONFIG_SMP > + { > + int cpu; > + > + for_each_online_cpu(cpu) { > + while (ipd->cpudata[cpu].irq_pending_hi != 0) > + cpu_relax(); > + } > + } > +#endif /* CONFIG_SMP */ > +} > + > +void __ipipe_unstall_root(void) > +{ > + ipipe_declare_cpuid; > + > + local_irq_disable_hw(); > + > + ipipe_load_cpuid(); > + > + __clear_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); > + > + if (ipipe_root_domain->cpudata[cpuid].irq_pending_hi != 0) > + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); > + > + local_irq_enable_hw(); > +} > + > +unsigned long __ipipe_test_root(void) > +{ > + unsigned long flags, x; > + ipipe_declare_cpuid; > + > + ipipe_get_cpu(flags); /* Care for migration. */ > + x = test_bit(IPIPE_STALL_FLAG, &ipipe_root_domain->cpudata[cpuid].status); > + ipipe_put_cpu(flags); > + > + return x; > +} > + > +unsigned long __ipipe_test_and_stall_root(void) > +{ > + unsigned long flags, x; > + ipipe_declare_cpuid; > + > + ipipe_get_cpu(flags); /* Care for migration. */ > + x = test_and_set_bit(IPIPE_STALL_FLAG, > + &ipipe_root_domain->cpudata[cpuid].status); > + ipipe_put_cpu(flags); > + > + return x; > +} > + > +void fastcall __ipipe_restore_root(unsigned long x) > +{ > + if (x) > + __ipipe_stall_root(); > + else > + __ipipe_unstall_root(); > +} > + > +void fastcall ipipe_stall_pipeline_from(struct ipipe_domain *ipd) > +{ > + ipipe_declare_cpuid; > +#ifdef CONFIG_SMP > + unsigned long flags; > + > + ipipe_lock_cpu(flags); /* Care for migration. */ > + > + __set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > + > + if (!__ipipe_pipeline_head_p(ipd)) > + ipipe_unlock_cpu(flags); > +#else /* CONFIG_SMP */ > + set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > + > + if (__ipipe_pipeline_head_p(ipd)) > + local_irq_disable_hw(); > +#endif /* CONFIG_SMP */ > +} > + > +unsigned long fastcall ipipe_test_and_stall_pipeline_from(struct ipipe_domain *ipd) > +{ > + ipipe_declare_cpuid; > + unsigned long s; > +#ifdef CONFIG_SMP > + unsigned long flags; > + > + ipipe_lock_cpu(flags); /* Care for migration. */ > + > + s = __test_and_set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > + > + if (!__ipipe_pipeline_head_p(ipd)) > + ipipe_unlock_cpu(flags); > +#else /* CONFIG_SMP */ > + s = test_and_set_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > + > + if (__ipipe_pipeline_head_p(ipd)) > + local_irq_disable_hw(); > +#endif /* CONFIG_SMP */ > + > + return s; > +} > + > +/* > + * ipipe_unstall_pipeline_from() -- Unstall the pipeline and > + * synchronize pending interrupts for a given domain. See > + * __ipipe_walk_pipeline() for more information. > + */ > +void fastcall ipipe_unstall_pipeline_from(struct ipipe_domain *ipd) > +{ > + struct list_head *pos; > + unsigned long flags; > + ipipe_declare_cpuid; > + > + ipipe_lock_cpu(flags); > + > + __clear_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > + > + if (ipd == ipipe_percpu_domain[cpuid]) > + pos = &ipd->p_link; > + else > + pos = __ipipe_pipeline.next; > + > + __ipipe_walk_pipeline(pos, cpuid); > + > + if (__ipipe_pipeline_head_p(ipd)) > + local_irq_enable_hw(); > + else > + ipipe_unlock_cpu(flags); > +} > + > +unsigned long fastcall ipipe_test_and_unstall_pipeline_from(struct ipipe_domain *ipd) > +{ > + unsigned long flags, x; > + ipipe_declare_cpuid; > + > + ipipe_get_cpu(flags); > + x = test_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpuid].status); > + ipipe_unstall_pipeline_from(ipd); > + ipipe_put_cpu(flags); > + > + return x; > +} > + > +void fastcall ipipe_restore_pipeline_from(struct ipipe_domain *ipd, > + unsigned long x) > +{ > + if (x) > + ipipe_stall_pipeline_from(ipd); > + else > + ipipe_unstall_pipeline_from(ipd); > +} > + > +void ipipe_unstall_pipeline_head(void) > +{ > + struct ipipe_domain *head; > + unsigned long flags; > + ipipe_declare_cpuid; > + > + ipipe_lock_cpu(flags); > + head = __ipipe_pipeline_head(); > + __clear_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); > + > + if (unlikely(head->cpudata[cpuid].irq_pending_hi != 0)) { > + if (likely(head == ipipe_percpu_domain[cpuid])) > + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); > + else > + __ipipe_walk_pipeline(&head->p_link, cpuid); > + } > + > + local_irq_enable_hw(); > +} > + > +void fastcall __ipipe_restore_pipeline_head(struct ipipe_domain *head, unsigned long x) > +{ > + ipipe_declare_cpuid; > + unsigned long flags; > + > + ipipe_lock_cpu(flags); > + > + if (x) { > +#ifdef CONFIG_DEBUG_KERNEL > + static int warned; > + if (!warned && test_and_set_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status)) { > + /* > + * Already stalled albeit ipipe_restore_pipeline_head() > + * should have detected it? Send a warning once.\n"); > + */ > + warned = 1; > + printk(KERN_WARNING > + "I-pipe: ipipe_restore_pipeline_head() optimization failed.\n"); > + dump_stack(); > + } > +#else /* !CONFIG_DEBUG_KERNEL */ > + set_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); > +#endif /* CONFIG_DEBUG_KERNEL */ > + } > + else { > + __clear_bit(IPIPE_STALL_FLAG, &head->cpudata[cpuid].status); > + if (unlikely(head->cpudata[cpuid].irq_pending_hi != 0)) { > + if (likely(head == ipipe_percpu_domain[cpuid])) > + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); > + else > + __ipipe_walk_pipeline(&head->p_link, cpuid); > + } > + local_irq_enable_hw(); > + } > +} > + > +/* __ipipe_walk_pipeline(): Plays interrupts pending in the log. Must > + be called with local hw interrupts disabled. */ > + > +void fastcall __ipipe_walk_pipeline(struct list_head *pos, int cpuid) > +{ > + struct ipipe_domain *this_domain = ipipe_percpu_domain[cpuid]; > + > + while (pos != &__ipipe_pipeline) { > + struct ipipe_domain *next_domain = > + list_entry(pos, struct ipipe_domain, p_link); > + > + if (test_bit > + (IPIPE_STALL_FLAG, &next_domain->cpudata[cpuid].status)) > + break; /* Stalled stage -- do not go further. */ > + > + if (next_domain->cpudata[cpuid].irq_pending_hi != 0) { > + > + if (next_domain == this_domain) > + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); > + else { > + __ipipe_switch_to(this_domain, next_domain, > + cpuid); > + > + ipipe_load_cpuid(); /* Processor might have changed. */ > + > + if (this_domain->cpudata[cpuid]. > + irq_pending_hi != 0 > + && !test_bit(IPIPE_STALL_FLAG, > + &this_domain->cpudata[cpuid].status)) > + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); > + } > + > + break; > + } else if (next_domain == this_domain) > + break; > + > + pos = next_domain->p_link.next; > + } > +} > + > +/* > + * ipipe_suspend_domain() -- Suspend the current domain, switching to > + * the next one which has pending work down the pipeline. > + */ > +void ipipe_suspend_domain(void) > +{ > + struct ipipe_domain *this_domain, *next_domain; > + struct list_head *ln; > + unsigned long flags; > + ipipe_declare_cpuid; > + > + ipipe_lock_cpu(flags); > + > + this_domain = next_domain = ipipe_percpu_domain[cpuid]; > + > + __clear_bit(IPIPE_STALL_FLAG, &this_domain->cpudata[cpuid].status); > + > + if (this_domain->cpudata[cpuid].irq_pending_hi != 0) > + goto sync_stage; > + > + for (;;) { > + ln = next_domain->p_link.next; > + > + if (ln == &__ipipe_pipeline) > + break; > + > + next_domain = list_entry(ln, struct ipipe_domain, p_link); > + > + if (test_bit(IPIPE_STALL_FLAG, > + &next_domain->cpudata[cpuid].status)) > + break; > + > + if (next_domain->cpudata[cpuid].irq_pending_hi == 0) > + continue; > + > + ipipe_percpu_domain[cpuid] = next_domain; > + > +sync_stage: > + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); > + > + ipipe_load_cpuid(); /* Processor might have changed. */ > + > + if (ipipe_percpu_domain[cpuid] != next_domain) > + /* > + * Something has changed the current domain under our > + * feet, recycling the register set; take note. > + */ > + this_domain = ipipe_percpu_domain[cpuid]; > + } > + > + ipipe_percpu_domain[cpuid] = this_domain; > + > + ipipe_unlock_cpu(flags); > +} > + > +/* ipipe_alloc_virq() -- Allocate a pipelined virtual/soft interrupt. > + * Virtual interrupts are handled in exactly the same way than their > + * hw-generated counterparts wrt pipelining. > + */ > +unsigned ipipe_alloc_virq(void) > +{ > + unsigned long flags, irq = 0; > + int ipos; > + > + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); > + > + if (__ipipe_virtual_irq_map != ~0) { > + ipos = ffz(__ipipe_virtual_irq_map); > + set_bit(ipos, &__ipipe_virtual_irq_map); > + irq = ipos + IPIPE_VIRQ_BASE; > + } > + > + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); > + > + return irq; > +} > + > +/* ipipe_virtualize_irq() -- Attach a handler (and optionally a hw > + acknowledge routine) to an interrupt for a given domain. */ > + > +int ipipe_virtualize_irq(struct ipipe_domain *ipd, > + unsigned irq, > + ipipe_irq_handler_t handler, > + void *cookie, > + ipipe_irq_ackfn_t acknowledge, > + unsigned modemask) > +{ > + unsigned long flags; > + int err; > + > + if (irq >= IPIPE_NR_IRQS) > + return -EINVAL; > + > + if (ipd->irqs[irq].control & IPIPE_SYSTEM_MASK) > + return -EPERM; > + > + if (!test_bit(IPIPE_AHEAD_FLAG, &ipd->flags)) > + /* Silently unwire interrupts for non-heading domains. */ > + modemask &= ~IPIPE_WIRED_MASK; > + > + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); > + > + if (handler != NULL) { > + > + if (handler == IPIPE_SAME_HANDLER) { > + handler = ipd->irqs[irq].handler; > + cookie = ipd->irqs[irq].cookie; > + > + if (handler == NULL) { > + err = -EINVAL; > + goto unlock_and_exit; > + } > + } else if ((modemask & IPIPE_EXCLUSIVE_MASK) != 0 && > + ipd->irqs[irq].handler != NULL) { > + err = -EBUSY; > + goto unlock_and_exit; > + } > + > + if ((modemask & (IPIPE_SHARED_MASK | IPIPE_PASS_MASK)) == > + IPIPE_SHARED_MASK) { > + err = -EINVAL; > + goto unlock_and_exit; > + } > + > + /* Wired interrupts can only be delivered to domains > + * always heading the pipeline. */ > + > + if ((modemask & IPIPE_WIRED_MASK) != 0) { > + if ((modemask & (IPIPE_SHARED_MASK | IPIPE_PASS_MASK | IPIPE_STICKY_MASK)) != 0) { > + err = -EINVAL; > + goto unlock_and_exit; > + } > + modemask |= (IPIPE_HANDLE_MASK); > + } > + > + if ((modemask & IPIPE_STICKY_MASK) != 0) > + modemask |= IPIPE_HANDLE_MASK; > + } else > + modemask &= > + ~(IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK | > + IPIPE_SHARED_MASK | IPIPE_EXCLUSIVE_MASK | IPIPE_WIRED_MASK); > + > + if (acknowledge == NULL) { > + if ((modemask & IPIPE_SHARED_MASK) == 0) { > + if (!ipipe_virtual_irq_p(irq)) { > + /* Acknowledge handler unspecified for a hw > + interrupt -- this is ok in non-shared > + management mode, but we will force the use > + of the Linux-defined handler instead. */ > + acknowledge = ipipe_root_domain->irqs[irq].acknowledge; > + } > + } > + else { > + /* A valid acknowledge handler to be called in shared mode > + is required when declaring a shared IRQ. */ > + err = -EINVAL; > + goto unlock_and_exit; > + } > + } > + > + ipd->irqs[irq].handler = handler; > + ipd->irqs[irq].cookie = cookie; > + ipd->irqs[irq].acknowledge = acknowledge; > + ipd->irqs[irq].control = modemask; > + > + if (irq < NR_IRQS && > + handler != NULL && > + !ipipe_virtual_irq_p(irq) && (modemask & IPIPE_ENABLE_MASK) != 0) { > + if (ipd != ipipe_current_domain) { > + /* IRQ enable/disable state is domain-sensitive, so we may > + not change it for another domain. What is allowed > + however is forcing some domain to handle an interrupt > + source, by passing the proper 'ipd' descriptor which > + thus may be different from ipipe_current_domain. */ > + err = -EPERM; > + goto unlock_and_exit; > + } > + > + __ipipe_enable_irq(irq); > + } > + > + err = 0; > + > + unlock_and_exit: > + > + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); > + > + return err; > +} > + > +/* ipipe_control_irq() -- Change modes of a pipelined interrupt for > + * the current domain. */ > + > +int ipipe_control_irq(unsigned irq, unsigned clrmask, unsigned setmask) > +{ > + struct ipipe_domain *ipd; > + unsigned long flags; > + > + if (irq >= IPIPE_NR_IRQS) > + return -EINVAL; > + > + ipd = ipipe_current_domain; > + > + if (ipd->irqs[irq].control & IPIPE_SYSTEM_MASK) > + return -EPERM; > + > + if (((setmask | clrmask) & IPIPE_SHARED_MASK) != 0) > + return -EINVAL; > + > + if (ipd->irqs[irq].handler == NULL) > + setmask &= ~(IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK); > + > + if ((setmask & IPIPE_STICKY_MASK) != 0) > + setmask |= IPIPE_HANDLE_MASK; > + > + if ((clrmask & (IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK)) != 0) /* If one goes, both go. */ > + clrmask |= (IPIPE_HANDLE_MASK | IPIPE_STICKY_MASK); > + > + spin_lock_irqsave_hw(&__ipipe_pipelock, flags); > + > + ipd->irqs[irq].control &= ~clrmask; > + ipd->irqs[irq].control |= setmask; > + > + if ((setmask & IPIPE_ENABLE_MASK) != 0) > + __ipipe_enable_irq(irq); > + else if ((clrmask & IPIPE_ENABLE_MASK) != 0) > + __ipipe_disable_irq(irq); > + > + spin_unlock_irqrestore_hw(&__ipipe_pipelock, flags); > + > + return 0; > +} > + > +/* __ipipe_dispatch_event() -- Low-level event dispatcher. */ > + > +int fastcall __ipipe_dispatch_event (unsigned event, void *data) > +{ > + struct ipipe_domain *start_domain, *this_domain, *next_domain; > + struct list_head *pos, *npos; > + unsigned long flags; > + ipipe_declare_cpuid; > + int propagate = 1; > + > + ipipe_lock_cpu(flags); > + > + start_domain = this_domain = ipipe_percpu_domain[cpuid]; > + > + list_for_each_safe(pos,npos,&__ipipe_pipeline) { > + > + next_domain = list_entry(pos,struct ipipe_domain,p_link); > + > + /* > + * Note: Domain migration may occur while running > + * event or interrupt handlers, in which case the > + * current register set is going to be recycled for a > + * different domain than the initiating one. We do > + * care for that, always tracking the current domain > + * descriptor upon return from those handlers. > + */ > + if (next_domain->evhand[event] != NULL) { > + ipipe_percpu_domain[cpuid] = next_domain; > + ipipe_unlock_cpu(flags); > + propagate = !next_domain->evhand[event](event,start_domain,data); > + ipipe_lock_cpu(flags); > + if (ipipe_percpu_domain[cpuid] != next_domain) > + this_domain = ipipe_percpu_domain[cpuid]; > + } > + > + if (next_domain != ipipe_root_domain && /* NEVER sync the root stage here. */ > + next_domain->cpudata[cpuid].irq_pending_hi != 0 && > + !test_bit(IPIPE_STALL_FLAG,&next_domain->cpudata[cpuid].status)) { > + ipipe_percpu_domain[cpuid] = next_domain; > + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); > + ipipe_load_cpuid(); > + if (ipipe_percpu_domain[cpuid] != next_domain) > + this_domain = ipipe_percpu_domain[cpuid]; > + } > + > + ipipe_percpu_domain[cpuid] = this_domain; > + > + if (next_domain == this_domain || !propagate) > + break; > + } > + > + ipipe_unlock_cpu(flags); > + > + return !propagate; > +} > + > +/* > + * __ipipe_dispatch_wired -- Wired interrupt dispatcher. Wired > + * interrupts are immediately and unconditionally delivered to the > + * domain heading the pipeline upon receipt, and such domain must have > + * been registered as an invariant head for the system (priority == > + * IPIPE_HEAD_PRIORITY). The motivation for using wired interrupts is > + * to get an extra-fast dispatching path for those IRQs, by relying on > + * a straightforward logic based on assumptions that must always be > + * true for invariant head domains. The following assumptions are > + * made when dealing with such interrupts: > + * > + * 1- Wired interrupts are purely dynamic, i.e. the decision to > + * propagate them down the pipeline must be done from the head domain > + * ISR. > + * 2- Wired interrupts cannot be shared or sticky. > + * 3- The root domain cannot be an invariant pipeline head, in > + * consequence of what the root domain cannot handle wired > + * interrupts. > + * 4- Wired interrupts must have a valid acknowledge handler for the > + * head domain (if needed), and in any case, must not rely on handlers > + * provided by lower priority domains during the acknowledge cycle > + * (see __ipipe_handle_irq). > + * > + * Called with hw interrupts off. > + */ > + > +int fastcall __ipipe_dispatch_wired(struct ipipe_domain *head, unsigned irq) > +{ > + struct ipcpudata *cpudata; > + struct ipipe_domain *old; > + ipipe_declare_cpuid; > + > + ipipe_load_cpuid(); > + cpudata = &head->cpudata[cpuid]; > + cpudata->irq_counters[irq].total_hits++; > + > + if (test_bit(IPIPE_LOCK_FLAG, &head->irqs[irq].control)) { > + /* If we can't process this IRQ right now, we must > + * mark it as pending, so that it will get played > + * during normal log sync when the corresponding > + * interrupt source is eventually unlocked. */ > + cpudata->irq_counters[irq].pending_hits++; > + return 0; > + } > + > + if (test_bit(IPIPE_STALL_FLAG, &cpudata->status)) { > + cpudata->irq_counters[irq].pending_hits++; > + __ipipe_set_irq_bit(head, cpuid, irq); > + return 0; > + } > + > + old = ipipe_percpu_domain[cpuid]; > + ipipe_percpu_domain[cpuid] = head; /* Switch to the head domain. */ > + > + __set_bit(IPIPE_STALL_FLAG, &cpudata->status); > + head->irqs[irq].handler(irq,head->irqs[irq].cookie); /* Call the ISR. */ > + __ipipe_run_irqtail(); > + __clear_bit(IPIPE_STALL_FLAG, &cpudata->status); > + > + /* We expect the caller to start a complete pipeline walk upon > + * return, so that propagated interrupts will get played. */ > + > + if (ipipe_percpu_domain[cpuid] == head) > + ipipe_percpu_domain[cpuid] = old; /* Back to the preempted domain. */ > + > + return 1; > +} > + > +/* > + * __ipipe_sync_stage() -- Flush the pending IRQs for the current > + * domain (and processor). This routine flushes the interrupt log > + * (see "Optimistic interrupt protection" from D. Stodolsky et al. for > + * more on the deferred interrupt scheme). Every interrupt that > + * occurred while the pipeline was stalled gets played. WARNING: > + * callers on SMP boxen should always check for CPU migration on > + * return of this routine. One can control the kind of interrupts > + * which are going to be sync'ed using the syncmask > + * parameter. IPIPE_IRQMASK_ANY plays them all, IPIPE_IRQMASK_VIRT > + * plays virtual interrupts only. > + * > + * This routine must be called with hw interrupts off. > + */ > +void fastcall __ipipe_sync_stage(unsigned long syncmask) > +{ > + unsigned long mask, submask; > + struct ipcpudata *cpudata; > + struct ipipe_domain *ipd; > + ipipe_declare_cpuid; > + int level, rank; > + unsigned irq; > + > + ipipe_load_cpuid(); > + ipd = ipipe_percpu_domain[cpuid]; > + cpudata = &ipd->cpudata[cpuid]; > + > + if (__test_and_set_bit(IPIPE_SYNC_FLAG, &cpudata->status)) > + return; > + > + /* > + * The policy here is to keep the dispatching code interrupt-free > + * by stalling the current stage. If the upper domain handler > + * (which we call) wants to re-enable interrupts while in a safe > + * portion of the code (e.g. SA_INTERRUPT flag unset for Linux's > + * sigaction()), it will have to unstall (then stall again before > + * returning to us!) the stage when it sees fit. > + */ > + while ((mask = (cpudata->irq_pending_hi & syncmask)) != 0) { > + level = __ipipe_ffnz(mask); > + __clear_bit(level, &cpudata->irq_pending_hi); > + > + while ((submask = cpudata->irq_pending_lo[level]) != 0) { > + rank = __ipipe_ffnz(submask); > + irq = (level << IPIPE_IRQ_ISHIFT) + rank; > + > + if (test_bit(IPIPE_LOCK_FLAG, &ipd->irqs[irq].control)) { > + __clear_bit(rank, &cpudata->irq_pending_lo[level]); > + continue; > + } > + > + if (--cpudata->irq_counters[irq].pending_hits == 0) > + __clear_bit(rank, &cpudata->irq_pending_lo[level]); > + > + __set_bit(IPIPE_STALL_FLAG, &cpudata->status); > + __ipipe_run_isr(ipd, irq, cpuid); > +#ifdef CONFIG_SMP > + { > + int _cpuid = ipipe_processor_id(); > + > + if (_cpuid != cpuid) { /* Handle CPU migration. */ > + /* > + * We expect any domain to clear the SYNC bit each > + * time it switches in a new task, so that preemptions > + * and/or CPU migrations (in the SMP case) over the > + * ISR do not lock out the log syncer for some > + * indefinite amount of time. In the Linux case, > + * schedule() handles this (see kernel/sched.c). For > + * this reason, we don't bother clearing it here for > + * the source CPU in the migration handling case, > + * since it must have scheduled another task in by > + * now. > + */ > + cpuid = _cpuid; > + cpudata = &ipd->cpudata[cpuid]; > + __set_bit(IPIPE_SYNC_FLAG, &cpudata->status); > + } > + } > +#endif /* CONFIG_SMP */ > + > + __clear_bit(IPIPE_STALL_FLAG, &cpudata->status); > + } > + } > + > + __clear_bit(IPIPE_SYNC_FLAG, &cpudata->status); > +} > + > +#ifdef CONFIG_PROC_FS > + > +#include <linux/proc_fs.h> > + > +struct proc_dir_entry *ipipe_proc_root; > + > +static int __ipipe_version_info_proc(char *page, > + char **start, > + off_t off, int count, int *eof, void *data) > +{ > + int len = sprintf(page, "%s\n", IPIPE_VERSION_STRING); > + > + len -= off; > + > + if (len <= off + count) > + *eof = 1; > + > + *start = page + off; > + > + if(len > count) > + len = count; > + > + if(len < 0) > + len = 0; > + > + return len; > +} > + > +static int __ipipe_common_info_proc(char *page, > + char **start, > + off_t off, int count, int *eof, void *data) > +{ > + struct ipipe_domain *ipd = (struct ipipe_domain *)data; > + unsigned long ctlbits; > + unsigned irq, _irq; > + char *p = page; > + int len; > + > + spin_lock(&__ipipe_pipelock); > + > + if (test_bit(IPIPE_AHEAD_FLAG,&ipd->flags)) > + p += sprintf(p, "Invariant head"); > + else > + p += sprintf(p, "Priority=%d", ipd->priority); > + > + p += sprintf(p, ", Id=0x%.8x\n", ipd->domid); > + > + irq = 0; > + > + while (irq < IPIPE_NR_IRQS) { > + ctlbits = > + (ipd->irqs[irq]. > + control & (IPIPE_HANDLE_MASK | IPIPE_PASS_MASK | > + IPIPE_STICKY_MASK | IPIPE_WIRED_MASK)); > + if (irq >= IPIPE_NR_XIRQS && !ipipe_virtual_irq_p(irq)) { > + /* > + * There might be a hole between the last external > + * IRQ and the first virtual one; skip it. > + */ > + irq++; > + continue; > + } > + > + if (ipipe_virtual_irq_p(irq) > + && !test_bit(irq - IPIPE_VIRQ_BASE, > + &__ipipe_virtual_irq_map)) { > + /* Non-allocated virtual IRQ; skip it. */ > + irq++; > + continue; > + } > + > + /* > + * Attempt to group consecutive IRQ numbers having the > + * same virtualization settings in a single line. > + */ > + > + _irq = irq; > + > + while (++_irq < IPIPE_NR_IRQS) { > + if (ipipe_virtual_irq_p(_irq) != > + ipipe_virtual_irq_p(irq) > + || (ipipe_virtual_irq_p(_irq) > + && !test_bit(_irq - IPIPE_VIRQ_BASE, > + &__ipipe_virtual_irq_map)) > + || ctlbits != (ipd->irqs[_irq]. > + control & (IPIPE_HANDLE_MASK | > + IPIPE_PASS_MASK | > + IPIPE_STICKY_MASK))) > + break; > + } > + > + if (_irq == irq + 1) > + p += sprintf(p, "irq%u: ", irq); > + else > + p += sprintf(p, "irq%u-%u: ", irq, _irq - 1); > + > + /* > + * Statuses are as follows: > + * o "accepted" means handled _and_ passed down the pipeline. > + * o "grabbed" means handled, but the interrupt might be > + * terminated _or_ passed down the pipeline depending on > + * what the domain handler asks for to the I-pipe. > + * o "wired" is basically the same as "grabbed", except that > + * the interrupt is unconditionally delivered to an invariant > + * pipeline head domain. > + * o "passed" means unhandled by the domain but passed > + * down the pipeline. > + * o "discarded" means unhandled and _not_ passed down the > + * pipeline. The interrupt merely disappears from the > + * current domain down to the end of the pipeline. > + */ > + if (ctlbits & IPIPE_HANDLE_MASK) { > + if (ctlbits & IPIPE_PASS_MASK) > + p += sprintf(p, "accepted"); > + else if (ctlbits & IPIPE_WIRED_MASK) > + p += sprintf(p, "wired"); > + else > + p += sprintf(p, "grabbed"); > + } else if (ctlbits & IPIPE_PASS_MASK) > + p += sprintf(p, "passed"); > + else > + p += sprintf(p, "discarded"); > + > + if (ctlbits & IPIPE_STICKY_MASK) > + p += sprintf(p, ", sticky"); > + > + if (ipipe_virtual_irq_p(irq)) > + p += sprintf(p, ", virtual"); > + > + p += sprintf(p, "\n"); > + > + irq = _irq; > + } > + > + spin_unlock(&__ipipe_pipelock); > + > + len = p - page; > + > + if (len <= off + count) > + *eof = 1; > + > + *start = page + off; > + > + len -= off; > + > + if (len > count) > + len = count; > + > + if (len < 0) > + len = 0; > + > + return len; > +} > + > +void __ipipe_add_domain_proc(struct ipipe_domain *ipd) > +{ > + create_proc_read_entry(ipd->name,0444,ipipe_proc_root,&__ipipe_common_info_proc,ipd); > +} > + > +void __ipipe_remove_domain_proc(struct ipipe_domain *ipd) > +{ > + remove_proc_entry(ipd->name,ipipe_proc_root); > +} > + > +void ipipe_init_proc(void) > +{ > + ipipe_proc_root = create_proc_entry("ipipe",S_IFDIR, 0); > + create_proc_read_entry("version",0444,ipipe_proc_root,&__ipipe_version_info_proc,NULL); > + __ipipe_init_trace_proc(); > + __ipipe_add_domain_proc(ipipe_root_domain); > +} > + > +#endif /* CONFIG_PROC_FS */ > + > +EXPORT_SYMBOL(ipipe_virtualize_irq); > +EXPORT_SYMBOL(ipipe_control_irq); > +EXPORT_SYMBOL(ipipe_suspend_domain); > +EXPORT_SYMBOL(ipipe_alloc_virq); > +EXPORT_SYMBOL(ipipe_percpu_domain); > +EXPORT_SYMBOL(ipipe_root_domain); > +EXPORT_SYMBOL(ipipe_stall_pipeline_from); > +EXPORT_SYMBOL(ipipe_test_and_stall_pipeline_from); > +EXPORT_SYMBOL(ipipe_unstall_pipeline_from); > +EXPORT_SYMBOL(ipipe_restore_pipeline_from); > +EXPORT_SYMBOL(ipipe_test_and_unstall_pipeline_from); > +EXPORT_SYMBOL(ipipe_unstall_pipeline_head); > +EXPORT_SYMBOL(__ipipe_restore_pipeline_head); > +EXPORT_SYMBOL(__ipipe_unstall_root); > +EXPORT_SYMBOL(__ipipe_stall_root); > +EXPORT_SYMBOL(__ipipe_restore_root); > +EXPORT_SYMBOL(__ipipe_test_and_stall_root); > +EXPORT_SYMBOL(__ipipe_test_root); > +EXPORT_SYMBOL(__ipipe_dispatch_event); > +EXPORT_SYMBOL(__ipipe_dispatch_wired); > +EXPORT_SYMBOL(__ipipe_sync_stage); > +EXPORT_SYMBOL(__ipipe_pipeline); > +EXPORT_SYMBOL(__ipipe_pipelock); > +EXPORT_SYMBOL(__ipipe_virtual_irq_map); > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/ipipe/generic.c linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/generic.c > --- linux-2.6.16.5-tcl1/kernel/ipipe/generic.c 1970-01-01 01:00:00.000000000 +0100 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/ipipe/generic.c 2006-07-16 15:01:24.000000000 +0200 > @@ -0,0 +1,396 @@ > +/* -*- linux-c -*- > + * linux/kernel/ipipe/generic.c > + * > + * Copyright (C) 2002-2005 Philippe Gerum. > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, > + * USA; either version 2 of the License, or (at your option) any later > + * version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. > + * > + * Architecture-independent I-PIPE services. > + */ > + > +#include <linux/version.h> > +#include <linux/module.h> > +#include <linux/init.h> > +#include <linux/kernel.h> > +#include <linux/sched.h> > +#ifdef CONFIG_PROC_FS > +#include <linux/proc_fs.h> > +#endif /* CONFIG_PROC_FS */ > + > +MODULE_DESCRIPTION("I-pipe"); > +MODULE_LICENSE("GPL"); > + > +static int __ipipe_ptd_key_count; > + > +static unsigned long __ipipe_ptd_key_map; > + > +/* ipipe_register_domain() -- Link a new domain to the pipeline. */ > + > +int ipipe_register_domain(struct ipipe_domain *ipd, > + struct ipipe_domain_attr *attr) > +{ > + struct list_head *pos; > + unsigned long flags; > + > + if (ipipe_current_domain != ipipe_root_domain) { > + printk(KERN_WARNING > + "I-pipe: Only the root domain may register a new domain.\n"); > + return -EPERM; > + } > + > + if (attr->priority == IPIPE_HEAD_PRIORITY && > + test_bit(IPIPE_AHEAD_FLAG,&__ipipe_pipeline_head()->flags)) > + return -EAGAIN; /* Cannot override current head. */ > + > + flags = ipipe_critical_enter(NULL); > + > + list_for_each(pos, &__ipipe_pipeline) { > + struct ipipe_domain *_ipd = > + list_entry(pos, struct ipipe_domain, p_link); > + if (_ipd->domid == attr->domid) > + break; > + } > + > + ipipe_critical_exit(flags); > + > + if (pos != &__ipipe_pipeline) > + /* A domain with the given id already exists -- fail. */ > + return -EBUSY; > + > + ipd->name = attr->name; > + ipd->domid = attr->domid; > + ipd->pdd = attr->pdd; > + ipd->flags = 0; > + > + if (attr->priority == IPIPE_HEAD_PRIORITY) { > + ipd->priority = INT_MAX; > + __set_bit(IPIPE_AHEAD_FLAG,&ipd->flags); > + } > + else > + ipd->priority = attr->priority; > + > + __ipipe_init_stage(ipd); > + > + INIT_LIST_HEAD(&ipd->p_link); > + > +#ifdef CONFIG_PROC_FS > + __ipipe_add_domain_proc(ipd); > +#endif /* CONFIG_PROC_FS */ > + > + flags = ipipe_critical_enter(NULL); > + > + list_for_each(pos, &__ipipe_pipeline) { > + struct ipipe_domain *_ipd = > + list_entry(pos, struct ipipe_domain, p_link); > + if (ipd->priority > _ipd->priority) > + break; > + } > + > + list_add_tail(&ipd->p_link, pos); > + > + ipipe_critical_exit(flags); > + > + printk(KERN_WARNING "I-pipe: Domain %s registered.\n", ipd->name); > + > + /* > + * Finally, allow the new domain to perform its initialization > + * chores. > + */ > + > + if (attr->entry != NULL) { > + ipipe_declare_cpuid; > + > + ipipe_lock_cpu(flags); > + > + ipipe_percpu_domain[cpuid] = ipd; > + attr->entry(); > + ipipe_percpu_domain[cpuid] = ipipe_root_domain; > + > + ipipe_load_cpuid(); /* Processor might have changed. */ > + > + if (ipipe_root_domain->cpudata[cpuid].irq_pending_hi != 0 && > + !test_bit(IPIPE_STALL_FLAG, > + &ipipe_root_domain->cpudata[cpuid].status)) > + __ipipe_sync_pipeline(IPIPE_IRQMASK_ANY); > + > + ipipe_unlock_cpu(flags); > + } > + > + return 0; > +} > + > +/* ipipe_unregister_domain() -- Remove a domain from the pipeline. */ > + > +int ipipe_unregister_domain(struct ipipe_domain *ipd) > +{ > + unsigned long flags; > + > + if (ipipe_current_domain != ipipe_root_domain) { > + printk(KERN_WARNING > + "I-pipe: Only the root domain may unregister a domain.\n"); > + return -EPERM; > + } > + > + if (ipd == ipipe_root_domain) { > + printk(KERN_WARNING > + "I-pipe: Cannot unregister the root domain.\n"); > + return -EPERM; > + } > +#ifdef CONFIG_SMP > + { > + int nr_cpus = num_online_cpus(), _cpuid; > + unsigned irq; > + > + /* > + * In the SMP case, wait for the logged events to drain on > + * other processors before eventually removing the domain > + * from the pipeline. > + */ > + > + ipipe_unstall_pipeline_from(ipd); > + > + flags = ipipe_critical_enter(NULL); > + > + for (irq = 0; irq < IPIPE_NR_IRQS; irq++) { > + clear_bit(IPIPE_HANDLE_FLAG, &ipd->irqs[irq].control); > + clear_bit(IPIPE_STICKY_FLAG, &ipd->irqs[irq].control); > + set_bit(IPIPE_PASS_FLAG, &ipd->irqs[irq].control); > + } > + > + ipipe_critical_exit(flags); > + > + for (_cpuid = 0; _cpuid < nr_cpus; _cpuid++) > + for (irq = 0; irq < IPIPE_NR_IRQS; irq++) > + while (ipd->cpudata[_cpuid].irq_counters[irq].pending_hits > 0) > + cpu_relax(); > + } > +#endif /* CONFIG_SMP */ > + > +#ifdef CONFIG_PROC_FS > + __ipipe_remove_domain_proc(ipd); > +#endif /* CONFIG_PROC_FS */ > + > + /* > + * Simply remove the domain from the pipeline and we are almost done. > + */ > + > + flags = ipipe_critical_enter(NULL); > + list_del_init(&ipd->p_link); > + ipipe_critical_exit(flags); > + > + __ipipe_cleanup_domain(ipd); > + > + printk(KERN_WARNING "I-pipe: Domain %s unregistered.\n", ipd->name); > + > + return 0; > +} > + > +/* > + * ipipe_propagate_irq() -- Force a given IRQ propagation on behalf of > + * a running interrupt handler to the next domain down the pipeline. > + * ipipe_schedule_irq() -- Does almost the same as above, but attempts > + * to pend the interrupt for the current domain first. > + */ > +int fastcall __ipipe_schedule_irq(unsigned irq, struct list_head *head) > +{ > + struct list_head *ln; > + unsigned long flags; > + ipipe_declare_cpuid; > + > + if (irq >= IPIPE_NR_IRQS || > + (ipipe_virtual_irq_p(irq) > + && !test_bit(irq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map))) > + return -EINVAL; > + > + ipipe_lock_cpu(flags); > + > + ln = head; > + > + while (ln != &__ipipe_pipeline) { > + struct ipipe_domain *ipd = > + list_entry(ln, struct ipipe_domain, p_link); > + > + if (test_bit(IPIPE_HANDLE_FLAG, &ipd->irqs[irq].control)) { > + ipd->cpudata[cpuid].irq_counters[irq].total_hits++; > + ipd->cpudata[cpuid].irq_counters[irq].pending_hits++; > + __ipipe_set_irq_bit(ipd, cpuid, irq); > + ipipe_unlock_cpu(flags); > + return 1; > + } > + > + ln = ipd->p_link.next; > + } > + > + ipipe_unlock_cpu(flags); > + > + return 0; > +} > + > +/* ipipe_free_virq() -- Release a virtual/soft interrupt. */ > + > +int ipipe_free_virq(unsigned virq) > +{ > + if (!ipipe_virtual_irq_p(virq)) > + return -EINVAL; > + > + clear_bit(virq - IPIPE_VIRQ_BASE, &__ipipe_virtual_irq_map); > + > + return 0; > +} > + > +void ipipe_init_attr(struct ipipe_domain_attr *attr) > +{ > + attr->name = "anon"; > + attr->domid = 1; > + attr->entry = NULL; > + attr->priority = IPIPE_ROOT_PRIO; > + attr->pdd = NULL; > +} > + > +/* > + * ipipe_catch_event() -- Interpose or remove an event handler for a > + * given domain. > + */ > +ipipe_event_handler_t ipipe_catch_event(struct ipipe_domain *ipd, > + unsigned event, > + ipipe_event_handler_t handler) > +{ > + ipipe_event_handler_t old_handler; > + int self = 0; > + > + if (event & IPIPE_EVENT_SELF) { > + event &= ~IPIPE_EVENT_SELF; > + self = 1; > + } > + > + if (event >= IPIPE_NR_EVENTS) > + return NULL; > + > + if (!(old_handler = xchg(&ipd->evhand[event],handler))) { > + if (handler) { > + if (self) > + ipd->evself |= (1LL << event); > + else > + __ipipe_event_monitors[event]++; > + } > + } > + else if (!handler) { > + if (ipd->evself & (1LL << event)) > + ipd->evself &= ~(1LL << event); > + else > + __ipipe_event_monitors[event]--; > + } else if ((ipd->evself & (1LL << event)) && !self) { > + __ipipe_event_monitors[event]++; > + ipd->evself &= ~(1LL << event); > + } else if (!(ipd->evself & (1LL << event)) && self) { > + __ipipe_event_monitors[event]--; > + ipd->evself |= (1LL << event); > + } > + > + return old_handler; > +} > + > +cpumask_t ipipe_set_irq_affinity (unsigned irq, cpumask_t cpumask) > +{ > +#ifdef CONFIG_SMP > + if (irq >= IPIPE_NR_XIRQS) > + /* Allow changing affinity of external IRQs only. */ > + return CPU_MASK_NONE; > + > + if (num_online_cpus() > 1) > + return __ipipe_set_irq_affinity(irq,cpumask); > +#endif /* CONFIG_SMP */ > + > + return CPU_MASK_NONE; > +} > + > +int fastcall ipipe_send_ipi (unsigned ipi, cpumask_t cpumask) > + > +{ > +#ifdef CONFIG_SMP > + return __ipipe_send_ipi(ipi,cpumask); > +#else /* !CONFIG_SMP */ > + return -EINVAL; > +#endif /* CONFIG_SMP */ > +} > + > +int ipipe_alloc_ptdkey (void) > +{ > + unsigned long flags; > + int key = -1; > + > + spin_lock_irqsave_hw(&__ipipe_pipelock,flags); > + > + if (__ipipe_ptd_key_count < IPIPE_ROOT_NPTDKEYS) { > + key = ffz(__ipipe_ptd_key_map); > + set_bit(key,&__ipipe_ptd_key_map); > + __ipipe_ptd_key_count++; > + } > + > + spin_unlock_irqrestore_hw(&__ipipe_pipelock,flags); > + > + return key; > +} > + > +int ipipe_free_ptdkey (int key) > +{ > + unsigned long flags; > + > + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) > + return -EINVAL; > + > + spin_lock_irqsave_hw(&__ipipe_pipelock,flags); > + > + if (test_and_clear_bit(key,&__ipipe_ptd_key_map)) > + __ipipe_ptd_key_count--; > + > + spin_unlock_irqrestore_hw(&__ipipe_pipelock,flags); > + > + return 0; > +} > + > +int fastcall ipipe_set_ptd (int key, void *value) > + > +{ > + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) > + return -EINVAL; > + > + current->ptd[key] = value; > + > + return 0; > +} > + > +void fastcall *ipipe_get_ptd (int key) > + > +{ > + if (key < 0 || key >= IPIPE_ROOT_NPTDKEYS) > + return NULL; > + > + return current->ptd[key]; > +} > + > +EXPORT_SYMBOL(ipipe_register_domain); > +EXPORT_SYMBOL(ipipe_unregister_domain); > +EXPORT_SYMBOL(ipipe_free_virq); > +EXPORT_SYMBOL(ipipe_init_attr); > +EXPORT_SYMBOL(ipipe_catch_event); > +EXPORT_SYMBOL(ipipe_alloc_ptdkey); > +EXPORT_SYMBOL(ipipe_free_ptdkey); > +EXPORT_SYMBOL(ipipe_set_ptd); > +EXPORT_SYMBOL(ipipe_get_ptd); > +EXPORT_SYMBOL(ipipe_set_irq_affinity); > +EXPORT_SYMBOL(ipipe_send_ipi); > +EXPORT_SYMBOL(__ipipe_schedule_irq); > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/irq/handle.c linux-2.6.16.5-tcl1-ipipe/kernel/irq/handle.c > --- linux-2.6.16.5-tcl1/kernel/irq/handle.c 2006-05-07 15:37:02.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/irq/handle.c 2006-07-16 15:01:24.000000000 +0200 > @@ -81,6 +81,17 @@ fastcall int handle_IRQ_event(unsigned i > { > int ret, retval = 0, status = 0; > > +#ifdef CONFIG_IPIPE > + /* > + * If processing a timer tick, pass the original regs as > + * collected during preemption and not our phony - always > + * kernel-originated - frame, so that we don't wreck the > + * profiling code. > + */ > + if (__ipipe_tick_irq == irq) > + regs = __ipipe_tick_regs + smp_processor_id(); > +#endif /* CONFIG_IPIPE */ > + > if (!(action->flags & SA_INTERRUPT)) > local_irq_enable(); > > @@ -117,16 +128,20 @@ fastcall unsigned int __do_IRQ(unsigned > /* > * No locking required for CPU-local interrupts: > */ > +#ifndef CONFIG_IPIPE > if (desc->handler->ack) > desc->handler->ack(irq); > +#endif /* !CONFIG_IPIPE */ > action_ret = handle_IRQ_event(irq, regs, desc->action); > desc->handler->end(irq); > return 1; > } > > spin_lock(&desc->lock); > +#ifndef CONFIG_IPIPE > if (desc->handler->ack) > desc->handler->ack(irq); > +#endif /* !CONFIG_IPIPE */ > /* > * REPLAY is when Linux resends an IRQ that was dropped earlier > * WAITING is used by probe to mark irqs that are being tested > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/printk.c linux-2.6.16.5-tcl1-ipipe/kernel/printk.c > --- linux-2.6.16.5-tcl1/kernel/printk.c 2006-07-15 20:06:03.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/printk.c 2006-07-16 15:01:24.000000000 +0200 > @@ -520,6 +520,78 @@ __attribute__((weak)) unsigned long long > * printf(3) > */ > > +#ifdef CONFIG_IPIPE > + > +static ipipe_spinlock_t __ipipe_printk_lock = IPIPE_SPIN_LOCK_UNLOCKED; > + > +static int __ipipe_printk_fill; > + > +static char __ipipe_printk_buf[__LOG_BUF_LEN]; > + > +void __ipipe_flush_printk (unsigned virq, void *cookie) > +{ > + char *p = __ipipe_printk_buf; > + int len, lmax, out = 0; > + unsigned long flags; > + > + goto start; > + > + do { > + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); > + start: > + lmax = __ipipe_printk_fill; > + while (out < lmax) { > + len = strlen(p) + 1; > + printk("%s",p); > + p += len; > + out += len; > + } > + spin_lock_irqsave_hw(&__ipipe_printk_lock,flags); > + } > + while (__ipipe_printk_fill != lmax); > + > + __ipipe_printk_fill = 0; > + > + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); > +} > + > +asmlinkage int printk(const char *fmt, ...) > +{ > + int r, fbytes, oldcount; > + unsigned long flags; > + va_list args; > + > + va_start(args, fmt); > + > + if (ipipe_current_domain == ipipe_root_domain || > + test_bit(IPIPE_SPRINTK_FLAG,&ipipe_current_domain->flags) || > + oops_in_progress) { > + r = vprintk(fmt, args); > + goto out; > + } > + > + spin_lock_irqsave_hw(&__ipipe_printk_lock,flags); > + > + oldcount = __ipipe_printk_fill; > + fbytes = __LOG_BUF_LEN - oldcount; > + > + if (fbytes > 1) { > + r = vscnprintf(__ipipe_printk_buf + __ipipe_printk_fill, > + fbytes, fmt, args) + 1; /* account for the null byte */ > + __ipipe_printk_fill += r; > + } else > + r = 0; > + > + spin_unlock_irqrestore_hw(&__ipipe_printk_lock,flags); > + > + if (oldcount == 0) > + ipipe_trigger_irq(__ipipe_printk_virq); > +out: > + va_end(args); > + > + return r; > +} > +#else /* !CONFIG_IPIPE */ > asmlinkage int printk(const char *fmt, ...) > { > va_list args; > @@ -531,6 +603,7 @@ asmlinkage int printk(const char *fmt, . > > return r; > } > +#endif /* CONFIG_IPIPE */ > > /* cpu currently holding logbuf_lock */ > static volatile unsigned int printk_cpu = UINT_MAX; > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/sched.c linux-2.6.16.5-tcl1-ipipe/kernel/sched.c > --- linux-2.6.16.5-tcl1/kernel/sched.c 2006-07-13 15:49:59.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/sched.c 2006-07-16 15:01:24.000000000 +0200 > @@ -2872,13 +2872,18 @@ asmlinkage void __sched schedule(void) > unsigned long run_time; > int cpu, idx, new_prio; > > +#ifdef CONFIG_IPIPE > + if (unlikely(ipipe_current_domain != ipipe_root_domain)) { > + goto need_resched; > + } > +#endif /* CONFIG_IPIPE */ > /* > * Test if we are atomic. Since do_exit() needs to call into > * schedule() atomically, we ignore that path for now. > * Otherwise, whine if we are scheduling when we should not be. > */ > if (likely(!current->exit_state)) { > - if (unlikely(in_atomic())) { > + if (unlikely(!(current->state & TASK_ATOMICSWITCH) && in_atomic())) { > printk(KERN_ERR "scheduling while atomic: " > "%s/0x%08x/%d\n", > current->comm, preempt_count(), current->pid); > @@ -2887,8 +2892,19 @@ asmlinkage void __sched schedule(void) > } > profile_hit(SCHED_PROFILING, __builtin_return_address(0)); > > + if (unlikely(current->state & TASK_ATOMICSWITCH)) { > + current->state &= ~TASK_ATOMICSWITCH; > + goto preemption_off; > + } > need_resched: > preempt_disable(); > +preemption_off: > +#ifdef CONFIG_IPIPE > + if (unlikely(ipipe_current_domain != ipipe_root_domain)) { > + preempt_enable(); > + return; > + } > +#endif /* CONFIG_IPIPE */ > prev = current; > release_kernel_lock(prev); > need_resched_nonpreemptible: > @@ -3027,6 +3043,8 @@ switch_tasks: > prepare_task_switch(rq, next); > prev = context_switch(rq, prev, next); > barrier(); > + if (task_hijacked(prev)) > + return; > /* > * this_rq must be evaluated again because prev may have moved > * CPUs since it called schedule(), thus the 'rq' on its stack > @@ -3059,6 +3077,11 @@ asmlinkage void __sched preempt_schedule > struct task_struct *task = current; > int saved_lock_depth; > #endif > +#ifdef CONFIG_IPIPE > + /* Do not reschedule over non-Linux domains. */ > + if (ipipe_current_domain != ipipe_root_domain) > + return; > +#endif /* CONFIG_IPIPE */ > /* > * If there is a non-zero preempt_count or interrupts are disabled, > * we do not want to preempt the current task. Just return.. > @@ -3697,6 +3720,7 @@ recheck: > deactivate_task(p, rq); > oldprio = p->prio; > __setscheduler(p, policy, param->sched_priority); > + ipipe_setsched_notify(p); > if (array) { > __activate_task(p, rq); > /* > @@ -6165,3 +6189,50 @@ void set_curr_task(int cpu, task_t *p) > } > > #endif > + > +#ifdef CONFIG_IPIPE > + > +int ipipe_setscheduler_root (struct task_struct *p, int policy, int prio) > +{ > + prio_array_t *array; > + unsigned long flags; > + runqueue_t *rq; > + int oldprio; > + > + rq = task_rq_lock(p, &flags); > + array = p->array; > + if (array) > + deactivate_task(p, rq); > + oldprio = p->prio; > + __setscheduler(p, policy, prio); > + if (array) { > + __activate_task(p, rq); > + if (task_running(rq, p)) { > + if (p->prio > oldprio) > + resched_task(rq->curr); > + } else if (TASK_PREEMPTS_CURR(p, rq)) > + resched_task(rq->curr); > + } > + task_rq_unlock(rq, &flags); > + > + return 0; > +} > + > +EXPORT_SYMBOL(ipipe_setscheduler_root); > + > +int ipipe_reenter_root (struct task_struct *prev, int policy, int prio) > +{ > + finish_task_switch(this_rq(), prev); > + if (reacquire_kernel_lock(current) < 0) > + ; > + preempt_enable_no_resched(); > + > + if (current->policy != policy || current->rt_priority != prio) > + return ipipe_setscheduler_root(current,policy,prio); > + > + return 0; > +} > + > +EXPORT_SYMBOL(ipipe_reenter_root); > + > +#endif /* CONFIG_IPIPE */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/kernel/signal.c linux-2.6.16.5-tcl1-ipipe/kernel/signal.c > --- linux-2.6.16.5-tcl1/kernel/signal.c 2006-07-13 15:49:59.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/kernel/signal.c 2006-07-16 15:01:24.000000000 +0200 > @@ -604,6 +604,7 @@ void signal_wake_up(struct task_struct * > unsigned int mask; > > set_tsk_thread_flag(t, TIF_SIGPENDING); > + ipipe_sigwake_notify(t); /* TIF_SIGPENDING must be set first. */ > > /* > * For SIGKILL, we want to wake it up in the stopped/traced case. > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/lib/smp_processor_id.c linux-2.6.16.5-tcl1-ipipe/lib/smp_processor_id.c > --- linux-2.6.16.5-tcl1/lib/smp_processor_id.c 2006-05-07 15:37:02.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/lib/smp_processor_id.c 2006-07-16 15:01:24.000000000 +0200 > @@ -13,6 +13,11 @@ unsigned int debug_smp_processor_id(void > int this_cpu = raw_smp_processor_id(); > cpumask_t this_mask; > > +#ifdef CONFIG_IPIPE > + if (ipipe_current_domain != ipipe_root_domain) > + return this_cpu; > +#endif /* CONFIG_IPIPE */ > + > if (likely(preempt_count)) > goto out; > > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/lib/spinlock_debug.c linux-2.6.16.5-tcl1-ipipe/lib/spinlock_debug.c > --- linux-2.6.16.5-tcl1/lib/spinlock_debug.c 2006-05-07 16:42:15.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/lib/spinlock_debug.c 2006-07-16 15:01:24.000000000 +0200 > @@ -10,6 +10,7 @@ > #include <linux/spinlock.h> > #include <linux/interrupt.h> > #include <linux/delay.h> > +#include <linux/module.h> > > static void spin_bug(spinlock_t *lock, const char *msg) > { > @@ -96,6 +97,8 @@ void _raw_spin_lock(spinlock_t *lock) > debug_spin_lock_after(lock); > } > > +EXPORT_SYMBOL(_raw_spin_lock); > + > int _raw_spin_trylock(spinlock_t *lock) > { > int ret = __raw_spin_trylock(&lock->raw_lock); > @@ -111,12 +114,16 @@ int _raw_spin_trylock(spinlock_t *lock) > return ret; > } > > +EXPORT_SYMBOL(_raw_spin_trylock); > + > void _raw_spin_unlock(spinlock_t *lock) > { > debug_spin_unlock(lock); > __raw_spin_unlock(&lock->raw_lock); > } > > +EXPORT_SYMBOL(_raw_spin_unlock); > + > static void rwlock_bug(rwlock_t *lock, const char *msg) > { > static long print_once = 1; > @@ -167,6 +174,8 @@ void _raw_read_lock(rwlock_t *lock) > __read_lock_debug(lock); > } > > +EXPORT_SYMBOL(_raw_read_lock); > + > int _raw_read_trylock(rwlock_t *lock) > { > int ret = __raw_read_trylock(&lock->raw_lock); > @@ -180,12 +189,16 @@ int _raw_read_trylock(rwlock_t *lock) > return ret; > } > > +EXPORT_SYMBOL(_raw_read_trylock); > + > void _raw_read_unlock(rwlock_t *lock) > { > RWLOCK_BUG_ON(lock->magic != RWLOCK_MAGIC, lock, "bad magic"); > __raw_read_unlock(&lock->raw_lock); > } > > +EXPORT_SYMBOL(_raw_read_unlock); > + > static inline void debug_write_lock_before(rwlock_t *lock) > { > RWLOCK_BUG_ON(lock->magic != RWLOCK_MAGIC, lock, "bad magic"); > @@ -241,6 +254,8 @@ void _raw_write_lock(rwlock_t *lock) > debug_write_lock_after(lock); > } > > +EXPORT_SYMBOL(_raw_write_lock); > + > int _raw_write_trylock(rwlock_t *lock) > { > int ret = __raw_write_trylock(&lock->raw_lock); > @@ -256,8 +271,12 @@ int _raw_write_trylock(rwlock_t *lock) > return ret; > } > > +EXPORT_SYMBOL(_raw_write_trylock); > + > void _raw_write_unlock(rwlock_t *lock) > { > debug_write_unlock(lock); > __raw_write_unlock(&lock->raw_lock); > } > + > +EXPORT_SYMBOL(_raw_write_unlock); > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1/mm/vmalloc.c linux-2.6.16.5-tcl1-ipipe/mm/vmalloc.c > --- linux-2.6.16.5-tcl1/mm/vmalloc.c 2006-05-07 15:37:03.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/mm/vmalloc.c 2006-07-16 15:01:24.000000000 +0200 > @@ -19,6 +19,7 @@ > > #include <asm/uaccess.h> > #include <asm/tlbflush.h> > +#include <asm/pgalloc.h> > > > DEFINE_RWLOCK(vmlist_lock); > @@ -148,10 +149,14 @@ int map_vm_area(struct vm_struct *area, > BUG_ON(addr >= end); > pgd = pgd_offset_k(addr); > do { > + pgd_t oldpgd; > + memcpy(&oldpgd,pgd,sizeof(pgd_t)); > next = pgd_addr_end(addr, end); > err = vmap_pud_range(pgd, addr, next, prot, pages); > if (err) > break; > + if (pgd_val(oldpgd) != pgd_val(*pgd)) > + set_pgdir(addr, *pgd); > } while (pgd++, addr = next, addr != end); > flush_cache_vmap((unsigned long) area->addr, end); > return err; > _______________________________________________ > Xenomai-core mailing list > Xenomai-core@domain.hid > https://mail.gna.org/listinfo/xenomai-core ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-31 8:34 ` Bart Jonkers @ 2006-07-31 9:20 ` Detlef Vollmann 2006-07-31 10:33 ` Bart Jonkers 0 siblings, 1 reply; 36+ messages in thread From: Detlef Vollmann @ 2006-07-31 9:20 UTC (permalink / raw) To: Bart Jonkers; +Cc: xenomai Hello, Bart Jonkers wrote: > When I boot the kernel with ipipe enabled, linux receives no interrupts > anymore. Any idea to solve this? What are the symtomps? Are you sure you get no interrupts (incl. timer) or e.g. just no GPIO interrupts? Detlef BTW, could you try not to quote the full patch next time? -- Detlef Vollmann vollmann engineering gmbh Linux and C++ for Embedded Systems http://www.vollmann.ch/ Linux for PXA270 Colibri module: http://www.vollmann.ch/en/colibri/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-31 9:20 ` Detlef Vollmann @ 2006-07-31 10:33 ` Bart Jonkers 2006-07-31 11:08 ` Detlef Vollmann 0 siblings, 1 reply; 36+ messages in thread From: Bart Jonkers @ 2006-07-31 10:33 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai On Mon, 2006-07-31 at 11:20 +0200, Detlef Vollmann wrote: > Hello, > > Bart Jonkers wrote: > > When I boot the kernel with ipipe enabled, linux receives no interrupts > > anymore. Any idea to solve this? > What are the symtomps? This is what i see when the kernel boots: Starting kernel ... Uncompressing Linux...................................................................................... done, booting the kernel. Linux version 2.6.16 (bart@domain.hid) (gcc version 3.4.2) #7 Mon Jul 31 11:46:19 CEST 2006 CPU: XScale-PXA255 [69052d06] revision 6 (ARMv5TE) Machine: Philips Xsilo Development Platform (aka Jobo Giga Vu Pro) Memory policy: ECC disabled, Data cache writeback Memory clock: 99.53MHz (*27) Run Mode clock: 199.07MHz (*2) Turbo Mode clock: 199.07MHz (*1.0, inactive) CPU0: D VIVT undefined 5 cache CPU0: I cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets CPU0: D cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets Built 1 zonelists Kernel command line: root=/dev/nfs rw ip=bootp console=ttyS0,115200 mem=64M PID hash table entries: 512 (order: 9, 8192 bytes) I-pipe 1.3-04: pipeline enabled. Console: colour dummy device 80x30 Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) Memory: 64MB = 64MB total Memory: 62080KB available (2232K code, 441K data, 88K init) Mount-cache hash table entries: 512 CPU: Testing write buffer coherency: ok NET: Registered protocol family 16 NetWinder Floating Point Emulator V0.97 (double precision) JFFS2 version 2.2. (NAND) (C) 2001-2003 Red Hat, Inc. io scheduler noop registered io scheduler anticipatory registered (default) io scheduler deadline registered io scheduler cfq registered Console: switching to colour frame buffer device 30x40 pxa2xx-uart.0: ttyS0 at MMIO 0x40100000 (irq = 15) is a FFUART pxa2xx-uart.1: ttyS1 at MMIO 0x40200000 (irq = 14) is a BTUART pxa2xx-uart.2: ttyS2 at MMIO 0x40700000 (irq = 13) is a STUART pxa2xx-uart.3: ttyS3 at MMIO 0x41600000 (irq = 0) is a HWUART smc91x.c: v1.1, sep 22 2004 by Nicolas Pitre <nico@domain.hid> eth0: SMC91C11xFD (rev 1) at c4862300 IRQ 36 DMA 8 [nowait] eth0: Ethernet addr: 08:00:3e:26:0a:53 Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx hda: probing with STATUS(0x50) instead of ALTSTATUS(0x0a) hda: IC25N040ATMR04-0, ATA DISK drive hdb: probing with STATUS(0x00) instead of ALTSTATUS(0x0a) hdb: probing with STATUS(0x00) instead of ALTSTATUS(0x0a) ide0 at 0xfb000000-0xfb000007,0xfb000020 on irq 35 hda: max request size: 512KiB hda: lost interrupt hda: lost interrupt hda: Host Protected Area detected. current capacity is 78140160 sectors (40007 MB) native capacity is 185074430006016 sectors (94758108163 MB) hda: lost interrupt hda: task_no_data_intr: status=0x51 { DriveReady SeekComplete Error } hda: task_no_data_intr: error=0x04 { DriveStatusError } ide: failed opcode was: 0x37 hda: 78140160 sectors (40007 MB) w/1740KiB Cache, CHS=16383/255/63 hda: lost interrupt hda: cache flushes supported hda:<4>hda: lost interrupt hda: lost interrupt hda: lost interrupt hda: lost interrupt hda: lost interrupt hda: lost interrupt hda: lost interrupt hda: lost interrupt hda1 hda: lost interrupt Probing Xsilo flash at physical address 0x00000000 (32-bit buswidth) xsilo-0: Found 2 x16 devices at 0x0 in 32-bit bank Intel/Sharp Extended Query Table at 0x0031 Using buffer write method cfi_cmdset_0001: Erase suspend on write enabled Registering xsilo-0 as whole device Advanced Linux Sound Architecture Driver Version 1.0.11rc2 (Wed Jan 04 08:57:20 2006 UTC). input: Touchscreen panel as /class/input/input0 ALSA device list: #0: pxa2xx-ac97 (Philips UCB1400) NET: Registered protocol family 2 IP route cache hash table entries: 1024 (order: 0, 4096 bytes) TCP established hash table entries: 4096 (order: 2, 16384 bytes) TCP bind hash table entries: 4096 (order: 2, 16384 bytes) TCP: Hash tables configured (established 4096 bind 4096) TCP reno registered TCP bic registered NET: Registered protocol family 1 eth0: link down Sending BOOTP requests ...... timed out! IP-Config: Reopening network devices... eth0: link down Sending BOOTP requests ...... timed out! IP-Config: Reopening network devices... eth0: link down Sending BOOTP requests ...... timed out! IP-Config: Reopening network devices... eth0: link down Sending BOOTP requests ...... timed out! IP-Config: Reopening network devices... eth0: link down The interrupt for the harddisk and the network chip are GPIO interrupts. > Are you sure you get no interrupts (incl. timer) or e.g. just no GPIO > interrupts? I added a kernel timer to the a driver of me witch does a printk every 10 seconds. The printk shows up so I think that Linux still gets its timer interrupts. I also added an interrupt handler on a button which is connected to GPIO pin. The interrupt handler should print something when it is executed. When I push the button a couple of times nothing happens. So GPIO interrupts doesn't seems to work. > > Detlef > > BTW, could you try not to quote the full patch next time? Sorry, my apologies. Bart > ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-31 10:33 ` Bart Jonkers @ 2006-07-31 11:08 ` Detlef Vollmann 2006-08-02 13:09 ` Gilles Chanteperdrix 0 siblings, 1 reply; 36+ messages in thread From: Detlef Vollmann @ 2006-07-31 11:08 UTC (permalink / raw) To: Bart Jonkers; +Cc: xenomai Bart Jonkers wrote: > On Mon, 2006-07-31 at 11:20 +0200, Detlef Vollmann wrote: > > Bart Jonkers wrote: > > > When I boot the kernel with ipipe enabled, linux receives no interrupts > > > anymore. Any idea to solve this? > > What are the symtomps? > hda: lost interrupt > Sending BOOTP requests ...... timed out! > The interrupt for the harddisk and the network chip are GPIO interrupts. This is what I suspected. > > Are you sure you get no interrupts (incl. timer) or e.g. just no GPIO > > interrupts? > > I added a kernel timer to the a driver of me witch does a printk every > 10 seconds. The printk shows up so I think that Linux still gets its > timer interrupts. And if you can type something on the serial console, you also get the UART interrupt. > I also added an interrupt handler on a button which is connected to GPIO > pin. The interrupt handler should print something when it is executed. > When I push the button a couple of times nothing happens. So GPIO > interrupts doesn't seems to work. Probably GPIO0 and GPIO1 work. I suspect it's the cascading interrupt that doesn't work. When I played around with an earlier version of ipipe, I found that the IRQ_GPIO_2_80 (IRQ 10 on PXA270) is masked, but never unmasked again. The problem is that for the cascading interrupt you need a special IPIPE handler, and I don't think there is currently one for PXA. How is that done on other machines/architectures? Detlef -- Detlef Vollmann vollmann engineering gmbh Linux and C++ for Embedded Systems http://www.vollmann.ch/ Linux for PXA270 Colibri module: http://www.vollmann.ch/en/colibri/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-31 11:08 ` Detlef Vollmann @ 2006-08-02 13:09 ` Gilles Chanteperdrix 2006-08-02 13:33 ` Bart Jonkers 0 siblings, 1 reply; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-08-02 13:09 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai Detlef Vollmann wrote: > > I also added an interrupt handler on a button which is connected to GPIO > > pin. The interrupt handler should print something when it is executed. > > When I push the button a couple of times nothing happens. So GPIO > > interrupts doesn't seems to work. > Probably GPIO0 and GPIO1 work. > I suspect it's the cascading interrupt that doesn't work. > When I played around with an earlier version of ipipe, I found that > the IRQ_GPIO_2_80 (IRQ 10 on PXA270) is masked, but never unmasked > again. > > The problem is that for the cascading interrupt you need a special > IPIPE handler, and I don't think there is currently one for PXA. > > How is that done on other machines/architectures? When looking at the ipipe_enable_pipeline function, we see that interrupts management routines are all intercepted by the I-pipe, so the cascaded interrupts management routines should be automatically intercepted. In order to have a better understanding of the issue, it would be interesting if you could trace the functions that are called on the path from the interrupt to the execution of the final handler. Re-reading the patch I sent, I also realised that an important part of the work is missing: the function gettimeoffset should be changed by the patch too. I would be surprised if your problem was related to this, though. -- Gilles Chanteperdrix. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-02 13:09 ` Gilles Chanteperdrix @ 2006-08-02 13:33 ` Bart Jonkers 2006-08-02 13:56 ` Gilles Chanteperdrix 2006-08-02 18:06 ` Gilles Chanteperdrix 0 siblings, 2 replies; 36+ messages in thread From: Bart Jonkers @ 2006-08-02 13:33 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai On Wed, 2006-08-02 at 15:09 +0200, Gilles Chanteperdrix wrote: > Detlef Vollmann wrote: > > > I also added an interrupt handler on a button which is connected to GPIO > > > pin. The interrupt handler should print something when it is executed. > > > When I push the button a couple of times nothing happens. So GPIO > > > interrupts doesn't seems to work. > > Probably GPIO0 and GPIO1 work. > > I suspect it's the cascading interrupt that doesn't work. > > When I played around with an earlier version of ipipe, I found that > > the IRQ_GPIO_2_80 (IRQ 10 on PXA270) is masked, but never unmasked > > again. > > > > The problem is that for the cascading interrupt you need a special > > IPIPE handler, and I don't think there is currently one for PXA. > > > > How is that done on other machines/architectures? > > When looking at the ipipe_enable_pipeline function, we see that > interrupts management routines are all intercepted by the I-pipe, so the > cascaded interrupts management routines should be automatically > intercepted. > > In order to have a better understanding of the issue, it would be > interesting if you could trace the functions that are called on the path > from the interrupt to the execution of the final handler. I have tracked and solved the issue. I have looked to the i.MX21 port and they added some code to the GPIO interrupt handler of the i.MX21. When IPIPE is active they unmask the interrupt for the GPIO pins at the end of the handler. I did the same for PXA and my GPIO interrupt problem is solved. I think that the same is needed for SA-1100. > > Re-reading the patch I sent, I also realised that an important part of > the work is missing: the function gettimeoffset should be changed by the > patch too. I would be surprised if your problem was related to this, > though. What need to be changed in the function gettimeoffset? Bart > ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-02 13:33 ` Bart Jonkers @ 2006-08-02 13:56 ` Gilles Chanteperdrix 2006-08-02 18:06 ` Gilles Chanteperdrix 1 sibling, 0 replies; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-08-02 13:56 UTC (permalink / raw) To: Bart Jonkers; +Cc: xenomai Bart Jonkers wrote: > > When looking at the ipipe_enable_pipeline function, we see that > > interrupts management routines are all intercepted by the I-pipe, so the > > cascaded interrupts management routines should be automatically > > intercepted. > > > > In order to have a better understanding of the issue, it would be > > interesting if you could trace the functions that are called on the path > > from the interrupt to the execution of the final handler. > > I have tracked and solved the issue. I have looked to the i.MX21 port > and they added some code to the GPIO interrupt handler of the i.MX21. > When IPIPE is active they unmask the interrupt for the GPIO pins at the > end of the handler. I did the same for PXA and my GPIO interrupt problem > is solved. I think that the same is needed for SA-1100. Could you send us an updated pxa-sa1100 patch ? > > > > Re-reading the patch I sent, I also realised that an important part of > > the work is missing: the function gettimeoffset should be changed by the > > patch too. I would be surprised if your problem was related to this, > > though. > What need to be changed in the function gettimeoffset? The function need to return the offset since the last Linux timer tick. With the additionnal constraint that the value returned by gettimeofday should be increasing over time. For example, a probably naive approach is to store the value of OSCR as the last timer tick date in a global variable in pxa_timer_interrupt, and to substract the current value of OSCR from this global variable in gettimeoffset. -- Gilles Chanteperdrix. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-02 13:33 ` Bart Jonkers 2006-08-02 13:56 ` Gilles Chanteperdrix @ 2006-08-02 18:06 ` Gilles Chanteperdrix 2006-08-03 9:12 ` Bart Jonkers 2006-08-03 11:23 ` Bart Jonkers 1 sibling, 2 replies; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-08-02 18:06 UTC (permalink / raw) To: Bart Jonkers; +Cc: xenomai [-- Attachment #1: message body and .signature --] [-- Type: text/plain, Size: 2140 bytes --] Bart Jonkers wrote: > On Wed, 2006-08-02 at 15:09 +0200, Gilles Chanteperdrix wrote: > > Detlef Vollmann wrote: > > > > I also added an interrupt handler on a button which is connected to GPIO > > > > pin. The interrupt handler should print something when it is executed. > > > > When I push the button a couple of times nothing happens. So GPIO > > > > interrupts doesn't seems to work. > > > Probably GPIO0 and GPIO1 work. > > > I suspect it's the cascading interrupt that doesn't work. > > > When I played around with an earlier version of ipipe, I found that > > > the IRQ_GPIO_2_80 (IRQ 10 on PXA270) is masked, but never unmasked > > > again. > > > > > > The problem is that for the cascading interrupt you need a special > > > IPIPE handler, and I don't think there is currently one for PXA. > > > > > > How is that done on other machines/architectures? > > > > When looking at the ipipe_enable_pipeline function, we see that > > interrupts management routines are all intercepted by the I-pipe, so the > > cascaded interrupts management routines should be automatically > > intercepted. > > > > In order to have a better understanding of the issue, it would be > > interesting if you could trace the functions that are called on the path > > from the interrupt to the execution of the final handler. > > I have tracked and solved the issue. I have looked to the i.MX21 port > and they added some code to the GPIO interrupt handler of the i.MX21. > When IPIPE is active they unmask the interrupt for the GPIO pins at the > end of the handler. I did the same for PXA and my GPIO interrupt problem > is solved. I think that the same is needed for SA-1100. Here is a new version of the ipipe-sa1100-pxa patch that unmaks interrupts at the end of the demux handlers, and that attempt to fix the gettimeoffset issue. I have run 20 minutes (time for OSCR to wrap) latency with a test program verifying that time returned by gettimeofday is never going backward. The patch and the test program are attached, it would be nice if you could test them. -- Gilles Chanteperdrix. [-- Attachment #2: ipipe-sa1100-pxa.2.patch --] [-- Type: text/plain, Size: 9947 bytes --] --- linux-2.6.16.5-tcl1/arch/arm/mach-pxa/irq.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-02 17:57:30.000000000 +0200 @@ -143,6 +143,10 @@ static struct irqchip pxa_low_gpio_chip static void pxa_gpio_demux_handler(unsigned int irq, struct irqdesc *desc, struct pt_regs *regs) { +#ifdef CONFIG_IPIPE + struct irqdesc *desc_unused = desc; + unsigned irq_unused = irq; +#endif /* CONFIG_IPIPE */ unsigned int mask; int loop; @@ -212,6 +216,10 @@ static void pxa_gpio_demux_handler(unsig } #endif } while (loop); + +#ifdef CONFIG_IPIPE + desc_unused->chip->unmask(irq_unused); +#endif /* CONFIG_IPIPE */ } static void pxa_ack_muxed_gpio(unsigned int irq) --- linux-2.6.16.5-tcl1/arch/arm/mach-pxa/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c 2006-08-02 19:17:30.000000000 +0200 @@ -19,6 +19,7 @@ #include <linux/signal.h> #include <linux/errno.h> #include <linux/sched.h> +#include <linux/module.h> #include <asm/system.h> #include <asm/hardware.h> @@ -30,6 +31,23 @@ #include <asm/arch/pxa-regs.h> +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int pxa_timer_initialized; +static unsigned long pxa_jiffies; +#endif /* CONFIG_IPIPE */ + static inline unsigned long pxa_get_rtc_time(void) { return RCNR; @@ -54,6 +72,9 @@ static unsigned long pxa_gettimeoffset ( { long ticks_to_match, elapsed, usec; +#ifdef CONFIG_IPIPE + if (!__ipipe_mach_timerstolen) { +#endif /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; @@ -63,6 +84,10 @@ static unsigned long pxa_gettimeoffset ( /* don't get fooled by the workaround in pxa_timer_interrupt() */ if (elapsed <= 0) return 0; +#ifdef CONFIG_IPIPE + } else + elapsed = OSCR - pxa_jiffies * LATCH; +#endif /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; @@ -105,9 +130,27 @@ pxa_timer_interrupt(int irq, void *dev_i * affect things only when the timer IRQ has been delayed by nearly * exactly one tick period which should be a pretty rare event. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) { + timer_tick(regs); + ++pxa_jiffies; + } else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); +#ifdef CONFIG_IPIPE + ++pxa_jiffies; +#else /* !CONFIG_IPIPE */ OSSR = OSSR_M0; /* Clear match on timer 0 */ +#endif /* !CONFIG_IPIPE */ next_match = (OSMR0 += LATCH); } while( (signed long)(next_match - OSCR) <= 8 ); @@ -139,6 +182,10 @@ static void __init pxa_timer_init(void) setup_irq(IRQ_OST0, &pxa_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + pxa_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -216,3 +263,69 @@ struct sys_timer pxa_timer = { .dyn_tick = &pxa_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(pxa_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + if (delay > 8) { + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); + } else + ipipe_trigger_irq(IRQ_OST0); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ --- linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/irq.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c 2006-08-02 17:55:48.000000000 +0200 @@ -111,6 +111,10 @@ static void sa1100_high_gpio_handler(unsigned int irq, struct irqdesc *desc, struct pt_regs *regs) { +#ifdef CONFIG_IPIPE + struct irqdesc *desc_unused = desc; + unsigned irq_unused = irq; +#endif /* CONFIG_IPIPE */ unsigned int mask; mask = GEDR & 0xfffff800; @@ -134,6 +138,10 @@ sa1100_high_gpio_handler(unsigned int ir mask = GEDR & 0xfffff800; } while (mask); + +#ifdef CONFIG_IPIPE + desc_unused->chip->unmask(irq_unused); +#endif /* CONFIG_IPIPE */ } /* --- linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c 2006-08-02 19:09:06.000000000 +0200 @@ -13,6 +13,7 @@ #include <linux/interrupt.h> #include <linux/timex.h> #include <linux/signal.h> +#include <linux/module.h> #include <asm/mach/time.h> #include <asm/hardware.h> @@ -20,6 +21,23 @@ #define RTC_DEF_DIVIDER (32768 - 1) #define RTC_DEF_TRIM 0 +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int sa1100_timer_initialized; +static unsigned long sa1100_jiffies; +#endif /* CONFIG_IPIPE */ + static unsigned long __init sa1100_get_rtc_time(void) { /* @@ -58,11 +76,18 @@ static unsigned long sa1100_gettimeoffse { unsigned long ticks_to_match, elapsed, usec; +#ifdef CONFIG_IPIPE + if (!__ipipe_mach_timerstolen) { +#endif /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; /* We need elapsed ticks since last match */ elapsed = LATCH - ticks_to_match; +#ifdef CONFIG_IPIPE + } else + elapsed = OSCR - sa1100_jiffies * LATCH; +#endif /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; @@ -97,9 +122,27 @@ sa1100_timer_interrupt(int irq, void *de * ensured, hence we can use do_gettimeofday() from interrupt * handlers. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) { + timer_tick(regs); + ++sa1100_jiffies; + } else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); +#ifdef CONFIG_IPIPE + ++sa1100_jiffies; +#else /* !CONFIG_IPIPE */ OSSR = OSSR_M0; /* Clear match on timer 0 */ +#endif /* !CONFIG_IPIPE */ next_match = (OSMR0 += LATCH); } while ((signed long)(next_match - OSCR) <= 0); @@ -131,6 +174,10 @@ static void __init sa1100_timer_init(voi setup_irq(IRQ_OST0, &sa1100_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + sa1100_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -209,3 +256,66 @@ struct sys_timer sa1100_timer = { .dyn_tick = &sa1100_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(sa1100_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ [-- Attachment #3: test_gettimeofday.c --] [-- Type: text/plain, Size: 338 bytes --] #include <stdio.h> #include <time.h> #include <sys/time.h> int main(void) { struct timeval tv, old_tv; gettimeofday(&old_tv, NULL); for (;;) { gettimeofday(&tv, NULL); if (tv.tv_sec < old_tv.tv_sec || (tv.tv_sec == old_tv.tv_sec && tv.tv_usec < old_tv.tv_usec)) printf("Time is going backward !\n"); old_tv = tv; } } ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-02 18:06 ` Gilles Chanteperdrix @ 2006-08-03 9:12 ` Bart Jonkers 2006-08-03 13:18 ` Gilles Chanteperdrix 2006-08-03 11:23 ` Bart Jonkers 1 sibling, 1 reply; 36+ messages in thread From: Bart Jonkers @ 2006-08-03 9:12 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai On Wed, 2006-08-02 at 20:06 +0200, Gilles Chanteperdrix wrote: > Bart Jonkers wrote: > > On Wed, 2006-08-02 at 15:09 +0200, Gilles Chanteperdrix wrote: > > > Detlef Vollmann wrote: > > > > > I also added an interrupt handler on a button which is connected to GPIO > > > > > pin. The interrupt handler should print something when it is executed. > > > > > When I push the button a couple of times nothing happens. So GPIO > > > > > interrupts doesn't seems to work. > > > > Probably GPIO0 and GPIO1 work. > > > > I suspect it's the cascading interrupt that doesn't work. > > > > When I played around with an earlier version of ipipe, I found that > > > > the IRQ_GPIO_2_80 (IRQ 10 on PXA270) is masked, but never unmasked > > > > again. > > > > > > > > The problem is that for the cascading interrupt you need a special > > > > IPIPE handler, and I don't think there is currently one for PXA. > > > > > > > > How is that done on other machines/architectures? > > > > > > When looking at the ipipe_enable_pipeline function, we see that > > > interrupts management routines are all intercepted by the I-pipe, so the > > > cascaded interrupts management routines should be automatically > > > intercepted. > > > > > > In order to have a better understanding of the issue, it would be > > > interesting if you could trace the functions that are called on the path > > > from the interrupt to the execution of the final handler. > > > > I have tracked and solved the issue. I have looked to the i.MX21 port > > and they added some code to the GPIO interrupt handler of the i.MX21. > > When IPIPE is active they unmask the interrupt for the GPIO pins at the > > end of the handler. I did the same for PXA and my GPIO interrupt problem > > is solved. I think that the same is needed for SA-1100. > > Here is a new version of the ipipe-sa1100-pxa patch that unmaks > interrupts at the end of the demux handlers, and that attempt to fix the > gettimeoffset issue. I have run 20 minutes (time for OSCR to wrap) latency > with a test program verifying that time returned by gettimeofday is > never going backward. > > The patch and the test program are attached, it would be nice if you > could test them. I have tested the patch and ran the test program. The test program give no problems. The interrupt problem in Linux is also solved. But I have another one. I have created a small real-time kernel module which uses the native interface to handle an GPIO interrupt. The problem is that Xenomai doesn't see the interrupt. When I use GPIO0 (which doesn't use the chained handler) to receive the interrupt everything works as it should. So it seems to be that xenomai have a problem with the chained handler. Does someone have an idea to solve this problem? Bart > > plain text document attachment (ipipe-sa1100-pxa.2.patch) > --- linux-2.6.16.5-tcl1/arch/arm/mach-pxa/irq.c 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-02 17:57:30.000000000 +0200 > @@ -143,6 +143,10 @@ static struct irqchip pxa_low_gpio_chip > static void pxa_gpio_demux_handler(unsigned int irq, struct irqdesc *desc, > struct pt_regs *regs) > { > +#ifdef CONFIG_IPIPE > + struct irqdesc *desc_unused = desc; > + unsigned irq_unused = irq; > +#endif /* CONFIG_IPIPE */ > unsigned int mask; > int loop; > > @@ -212,6 +216,10 @@ static void pxa_gpio_demux_handler(unsig > } > #endif > } while (loop); > + > +#ifdef CONFIG_IPIPE > + desc_unused->chip->unmask(irq_unused); > +#endif /* CONFIG_IPIPE */ > } > > static void pxa_ack_muxed_gpio(unsigned int irq) > --- linux-2.6.16.5-tcl1/arch/arm/mach-pxa/time.c 2006-05-07 15:36:35.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c 2006-08-02 19:17:30.000000000 +0200 > @@ -19,6 +19,7 @@ > #include <linux/signal.h> > #include <linux/errno.h> > #include <linux/sched.h> > +#include <linux/module.h> > > #include <asm/system.h> > #include <asm/hardware.h> > @@ -30,6 +31,23 @@ > #include <asm/arch/pxa-regs.h> > > > +#ifdef CONFIG_IPIPE > +#ifdef CONFIG_NO_IDLE_HZ > +#error "dynamic tick timer not yet supported with IPIPE" > +#endif /* CONFIG_NO_IDLE_HZ */ > +int __ipipe_mach_timerint = IRQ_OST0; > +EXPORT_SYMBOL(__ipipe_mach_timerint); > + > +int __ipipe_mach_timerstolen = 0; > +EXPORT_SYMBOL(__ipipe_mach_timerstolen); > + > +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; > +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); > + > +static int pxa_timer_initialized; > +static unsigned long pxa_jiffies; > +#endif /* CONFIG_IPIPE */ > + > static inline unsigned long pxa_get_rtc_time(void) > { > return RCNR; > @@ -54,6 +72,9 @@ static unsigned long pxa_gettimeoffset ( > { > long ticks_to_match, elapsed, usec; > > +#ifdef CONFIG_IPIPE > + if (!__ipipe_mach_timerstolen) { > +#endif > /* Get ticks before next timer match */ > ticks_to_match = OSMR0 - OSCR; > > @@ -63,6 +84,10 @@ static unsigned long pxa_gettimeoffset ( > /* don't get fooled by the workaround in pxa_timer_interrupt() */ > if (elapsed <= 0) > return 0; > +#ifdef CONFIG_IPIPE > + } else > + elapsed = OSCR - pxa_jiffies * LATCH; > +#endif > > /* Now convert them to usec */ > usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; > @@ -105,9 +130,27 @@ pxa_timer_interrupt(int irq, void *dev_i > * affect things only when the timer IRQ has been delayed by nearly > * exactly one tick period which should be a pretty rare event. > */ > +#ifdef CONFIG_IPIPE > + /* > + * - if Linux is running natively (no ipipe), ack and reprogram the timer > + * - if Linux is running under ipipe, but it still has the control over > + * the timer (no Xenomai for example), then reprogram the timer (ipipe > + * has already acked it) > + * - if some other domain has taken over the timer, then do nothing > + * (ipipe has acked it, and the other domain has reprogramed it) > + */ > + if (__ipipe_mach_timerstolen) { > + timer_tick(regs); > + ++pxa_jiffies; > + } else > +#endif /* CONFIG_IPIPE */ > do { > timer_tick(regs); > +#ifdef CONFIG_IPIPE > + ++pxa_jiffies; > +#else /* !CONFIG_IPIPE */ > OSSR = OSSR_M0; /* Clear match on timer 0 */ > +#endif /* !CONFIG_IPIPE */ > next_match = (OSMR0 += LATCH); > } while( (signed long)(next_match - OSCR) <= 8 ); > > @@ -139,6 +182,10 @@ static void __init pxa_timer_init(void) > setup_irq(IRQ_OST0, &pxa_timer_irq); > OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ > OSCR = 0; /* initialize free-running timer */ > + > +#ifdef CONFIG_IPIPE > + pxa_timer_initialized = 1; > +#endif /* CONFIG_IPIPE */ > } > > #ifdef CONFIG_NO_IDLE_HZ > @@ -216,3 +263,69 @@ struct sys_timer pxa_timer = { > .dyn_tick = &pxa_dyn_tick, > #endif > }; > + > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_acktimer(void) > +{ > + OSSR = OSSR_M0; /* Clear match on timer 0 */ > +} > + > +unsigned long long __ipipe_mach_get_tsc(void) > +{ > + if (likely(pxa_timer_initialized)) { > + static union { > +#ifdef __BIG_ENDIAN > + struct { > + unsigned long high; > + unsigned long low; > + }; > +#else /* __LITTLE_ENDIAN */ > + struct { > + unsigned long low; > + unsigned long high; > + }; > +#endif /* __LITTLE_ENDIAN */ > + unsigned long long full; > + } tsc[NR_CPUS], *local_tsc; > + unsigned long stamp, flags; > + unsigned long long result; > + > + local_irq_save_hw(flags); > + local_tsc = &tsc[ipipe_processor_id()]; > + stamp = OSCR; > + if (unlikely(stamp < local_tsc->low)) > + /* 32 bit counter wrapped, increment high word. */ > + local_tsc->high++; > + local_tsc->low = stamp; > + result = local_tsc->full; > + local_irq_restore_hw(flags); > + > + return result; > + } > + > + return 0; > +} > +EXPORT_SYMBOL(__ipipe_mach_get_tsc); > + > +/* > + * Reprogram the timer > + */ > + > +void __ipipe_mach_set_dec(unsigned long delay) > +{ > + if (delay > 8) { > + unsigned long flags; > + > + local_irq_save_hw(flags); > + OSMR0 = delay + OSCR; > + local_irq_restore_hw(flags); > + } else > + ipipe_trigger_irq(IRQ_OST0); > +} > +EXPORT_SYMBOL(__ipipe_mach_set_dec); > + > +unsigned long __ipipe_mach_get_dec(void) > +{ > + return OSMR0 - OSCR; > +} > +#endif /* CONFIG_IPIPE */ > --- linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/irq.c 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c 2006-08-02 17:55:48.000000000 +0200 > @@ -111,6 +111,10 @@ static void > sa1100_high_gpio_handler(unsigned int irq, struct irqdesc *desc, > struct pt_regs *regs) > { > +#ifdef CONFIG_IPIPE > + struct irqdesc *desc_unused = desc; > + unsigned irq_unused = irq; > +#endif /* CONFIG_IPIPE */ > unsigned int mask; > > mask = GEDR & 0xfffff800; > @@ -134,6 +138,10 @@ sa1100_high_gpio_handler(unsigned int ir > > mask = GEDR & 0xfffff800; > } while (mask); > + > +#ifdef CONFIG_IPIPE > + desc_unused->chip->unmask(irq_unused); > +#endif /* CONFIG_IPIPE */ > } > > /* > --- linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/time.c 2006-05-07 15:36:35.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c 2006-08-02 19:09:06.000000000 +0200 > @@ -13,6 +13,7 @@ > #include <linux/interrupt.h> > #include <linux/timex.h> > #include <linux/signal.h> > +#include <linux/module.h> > > #include <asm/mach/time.h> > #include <asm/hardware.h> > @@ -20,6 +21,23 @@ > #define RTC_DEF_DIVIDER (32768 - 1) > #define RTC_DEF_TRIM 0 > > +#ifdef CONFIG_IPIPE > +#ifdef CONFIG_NO_IDLE_HZ > +#error "dynamic tick timer not yet supported with IPIPE" > +#endif /* CONFIG_NO_IDLE_HZ */ > +int __ipipe_mach_timerint = IRQ_OST0; > +EXPORT_SYMBOL(__ipipe_mach_timerint); > + > +int __ipipe_mach_timerstolen = 0; > +EXPORT_SYMBOL(__ipipe_mach_timerstolen); > + > +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; > +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); > + > +static int sa1100_timer_initialized; > +static unsigned long sa1100_jiffies; > +#endif /* CONFIG_IPIPE */ > + > static unsigned long __init sa1100_get_rtc_time(void) > { > /* > @@ -58,11 +76,18 @@ static unsigned long sa1100_gettimeoffse > { > unsigned long ticks_to_match, elapsed, usec; > > +#ifdef CONFIG_IPIPE > + if (!__ipipe_mach_timerstolen) { > +#endif > /* Get ticks before next timer match */ > ticks_to_match = OSMR0 - OSCR; > > /* We need elapsed ticks since last match */ > elapsed = LATCH - ticks_to_match; > +#ifdef CONFIG_IPIPE > + } else > + elapsed = OSCR - sa1100_jiffies * LATCH; > +#endif > > /* Now convert them to usec */ > usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; > @@ -97,9 +122,27 @@ sa1100_timer_interrupt(int irq, void *de > * ensured, hence we can use do_gettimeofday() from interrupt > * handlers. > */ > +#ifdef CONFIG_IPIPE > + /* > + * - if Linux is running natively (no ipipe), ack and reprogram the timer > + * - if Linux is running under ipipe, but it still has the control over > + * the timer (no Xenomai for example), then reprogram the timer (ipipe > + * has already acked it) > + * - if some other domain has taken over the timer, then do nothing > + * (ipipe has acked it, and the other domain has reprogramed it) > + */ > + if (__ipipe_mach_timerstolen) { > + timer_tick(regs); > + ++sa1100_jiffies; > + } else > +#endif /* CONFIG_IPIPE */ > do { > timer_tick(regs); > +#ifdef CONFIG_IPIPE > + ++sa1100_jiffies; > +#else /* !CONFIG_IPIPE */ > OSSR = OSSR_M0; /* Clear match on timer 0 */ > +#endif /* !CONFIG_IPIPE */ > next_match = (OSMR0 += LATCH); > } while ((signed long)(next_match - OSCR) <= 0); > > @@ -131,6 +174,10 @@ static void __init sa1100_timer_init(voi > setup_irq(IRQ_OST0, &sa1100_timer_irq); > OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ > OSCR = 0; /* initialize free-running timer */ > + > +#ifdef CONFIG_IPIPE > + sa1100_timer_initialized = 1; > +#endif /* CONFIG_IPIPE */ > } > > #ifdef CONFIG_NO_IDLE_HZ > @@ -209,3 +256,66 @@ struct sys_timer sa1100_timer = { > .dyn_tick = &sa1100_dyn_tick, > #endif > }; > + > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_acktimer(void) > +{ > + OSSR = OSSR_M0; /* Clear match on timer 0 */ > +} > + > +unsigned long long __ipipe_mach_get_tsc(void) > +{ > + if (likely(sa1100_timer_initialized)) { > + static union { > +#ifdef __BIG_ENDIAN > + struct { > + unsigned long high; > + unsigned long low; > + }; > +#else /* __LITTLE_ENDIAN */ > + struct { > + unsigned long low; > + unsigned long high; > + }; > +#endif /* __LITTLE_ENDIAN */ > + unsigned long long full; > + } tsc[NR_CPUS], *local_tsc; > + unsigned long stamp, flags; > + unsigned long long result; > + > + local_irq_save_hw(flags); > + local_tsc = &tsc[ipipe_processor_id()]; > + stamp = OSCR; > + if (unlikely(stamp < local_tsc->low)) > + /* 32 bit counter wrapped, increment high word. */ > + local_tsc->high++; > + local_tsc->low = stamp; > + result = local_tsc->full; > + local_irq_restore_hw(flags); > + > + return result; > + } > + > + return 0; > +} > +EXPORT_SYMBOL(__ipipe_mach_get_tsc); > + > +/* > + * Reprogram the timer > + */ > + > +void __ipipe_mach_set_dec(unsigned long delay) > +{ > + unsigned long flags; > + > + local_irq_save_hw(flags); > + OSMR0 = delay + OSCR; > + local_irq_restore_hw(flags); > +} > +EXPORT_SYMBOL(__ipipe_mach_set_dec); > + > +unsigned long __ipipe_mach_get_dec(void) > +{ > + return OSMR0 - OSCR; > +} > +#endif /* CONFIG_IPIPE */ > plain text document attachment (test_gettimeofday.c) > #include <stdio.h> > #include <time.h> > #include <sys/time.h> > > int main(void) > { > struct timeval tv, old_tv; > > gettimeofday(&old_tv, NULL); > for (;;) { > gettimeofday(&tv, NULL); > if (tv.tv_sec < old_tv.tv_sec || > (tv.tv_sec == old_tv.tv_sec && tv.tv_usec < old_tv.tv_usec)) > printf("Time is going backward !\n"); > old_tv = tv; > } > } ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-03 9:12 ` Bart Jonkers @ 2006-08-03 13:18 ` Gilles Chanteperdrix 2006-08-03 14:56 ` Philippe Gerum 0 siblings, 1 reply; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-08-03 13:18 UTC (permalink / raw) To: Bart Jonkers; +Cc: xenomai Bart Jonkers wrote: > > Here is a new version of the ipipe-sa1100-pxa patch that unmaks > > interrupts at the end of the demux handlers, and that attempt to fix the > > gettimeoffset issue. I have run 20 minutes (time for OSCR to wrap) latency > > with a test program verifying that time returned by gettimeofday is > > never going backward. > > > > The patch and the test program are attached, it would be nice if you > > could test them. > I have tested the patch and ran the test program. > The test program give no problems. > > The interrupt problem in Linux is also solved. But I have another one. > I have created a small real-time kernel module which uses the native > interface to handle an GPIO interrupt. The problem is that Xenomai > doesn't see the interrupt. When I use GPIO0 (which doesn't use the > chained handler) to receive the interrupt everything works as it should. > > So it seems to be that xenomai have a problem with the chained handler. > Does someone have an idea to solve this problem? The problem is that the demux handler is a root domain handler, and directly call the cascaded interrupts root domain handlers through desc_handle_irq, whereas in order for the I-pipe to intercept the interrupt for any domain, the demux handler should be run for any domain, and invoke __ipipe_grab_irq, __ipipe_handle_irq or ipipe_trigger_irq so that the cascaded interrupts are pipelined. So, there are two possible fixes: - either fix the I-pipe patch so that the demux handler is invoked when the multiplexed interrupt is received for any domain, and triggers interrupts through ipipe_trigger_irq; - or you may fix the issue locally by registering a demux handler in Xenomai domain through ipipe_virtualize_irq_from, and in this demux handler only clear the interrupt you want and call ipipe_trigger_irq for them, and ipipe_propagate_irq the multiplexing interrupt to let other irqs be handled in the root domain. Philippe, is there a way to tell the I-ipipe that an interrupt handler should be executed for any domain? Is this what a "wired" handler is? -- Gilles Chanteperdrix. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-03 13:18 ` Gilles Chanteperdrix @ 2006-08-03 14:56 ` Philippe Gerum 2006-08-03 17:14 ` Gilles Chanteperdrix 0 siblings, 1 reply; 36+ messages in thread From: Philippe Gerum @ 2006-08-03 14:56 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai On Thu, 2006-08-03 at 15:18 +0200, Gilles Chanteperdrix wrote: > Bart Jonkers wrote: > > > Here is a new version of the ipipe-sa1100-pxa patch that unmaks > > > interrupts at the end of the demux handlers, and that attempt to fix the > > > gettimeoffset issue. I have run 20 minutes (time for OSCR to wrap) latency > > > with a test program verifying that time returned by gettimeofday is > > > never going backward. > > > > > > The patch and the test program are attached, it would be nice if you > > > could test them. > > I have tested the patch and ran the test program. > > The test program give no problems. > > > > The interrupt problem in Linux is also solved. But I have another one. > > I have created a small real-time kernel module which uses the native > > interface to handle an GPIO interrupt. The problem is that Xenomai > > doesn't see the interrupt. When I use GPIO0 (which doesn't use the > > chained handler) to receive the interrupt everything works as it should. > > > > So it seems to be that xenomai have a problem with the chained handler. > > Does someone have an idea to solve this problem? > > The problem is that the demux handler is a root domain handler, and > directly call the cascaded interrupts root domain handlers through > desc_handle_irq, whereas in order for the I-pipe to intercept the > interrupt for any domain, the demux handler should be run for any > domain, and invoke __ipipe_grab_irq, __ipipe_handle_irq or > ipipe_trigger_irq so that the cascaded interrupts are pipelined. > > So, there are two possible fixes: > - either fix the I-pipe patch so that the demux handler is invoked when > the multiplexed interrupt is received for any domain, and triggers > interrupts through ipipe_trigger_irq; Fixing the I-pipe is the way to go, definitely. If I understand this correctly, the best way to handle the demux case is to implement an I-pipe variant of the demultiplexing code currently available in pxa_gpio_demux_handler(), which would be called from __ipipe_grab_irq() when processing the multiplexed interrupt channel. The I-pipe-specific demux code would then log the decoded IRQ by a call to __ipipe_handle_irq() instead of running the root handler directly (i.e. desc_handle_irq), which would in turn preserve the ability to "wire" GPIO interrupts and also handle IRQ stickiness issues. This would resemble what we are doing right now on the powerpc arch, pulling the received IRQ number from a request to the machine-dependent layer. I'd see no issue in duplicating such handling since this is a rather short piece of invariant, hw-specific code which would not depend on Linux internals, but rather on the I-pipe's. > - or you may fix the issue locally by registering a demux handler in > Xenomai domain through ipipe_virtualize_irq_from, and in this demux > handler only clear the interrupt you want and call ipipe_trigger_irq > for them, and ipipe_propagate_irq the multiplexing interrupt to let > other irqs be handled in the root domain. > > Philippe, is there a way to tell the I-ipipe that an interrupt handler > should be executed for any domain? Nope, there is no interrupt handlers implicitely shared among domains in Adeos. > Is this what a "wired" handler is? > A wired interrupt handler is specifically associated to the domain which is know to always lead the pipeline, so that certain assumptions which we rely on to speed up the interrupt delivery are always true. -- Philippe. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-03 14:56 ` Philippe Gerum @ 2006-08-03 17:14 ` Gilles Chanteperdrix 2006-08-04 6:10 ` Bart Jonkers ` (2 more replies) 0 siblings, 3 replies; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-08-03 17:14 UTC (permalink / raw) To: rpm; +Cc: xenomai [-- Attachment #1: message body and .signature --] [-- Type: text/plain, Size: 3083 bytes --] Philippe Gerum wrote: > On Thu, 2006-08-03 at 15:18 +0200, Gilles Chanteperdrix wrote: > > Bart Jonkers wrote: > > > > Here is a new version of the ipipe-sa1100-pxa patch that unmaks > > > > interrupts at the end of the demux handlers, and that attempt to fix the > > > > gettimeoffset issue. I have run 20 minutes (time for OSCR to wrap) latency > > > > with a test program verifying that time returned by gettimeofday is > > > > never going backward. > > > > > > > > The patch and the test program are attached, it would be nice if you > > > > could test them. > > > I have tested the patch and ran the test program. > > > The test program give no problems. > > > > > > The interrupt problem in Linux is also solved. But I have another one. > > > I have created a small real-time kernel module which uses the native > > > interface to handle an GPIO interrupt. The problem is that Xenomai > > > doesn't see the interrupt. When I use GPIO0 (which doesn't use the > > > chained handler) to receive the interrupt everything works as it should. > > > > > > So it seems to be that xenomai have a problem with the chained handler. > > > Does someone have an idea to solve this problem? > > > > The problem is that the demux handler is a root domain handler, and > > directly call the cascaded interrupts root domain handlers through > > desc_handle_irq, whereas in order for the I-pipe to intercept the > > interrupt for any domain, the demux handler should be run for any > > domain, and invoke __ipipe_grab_irq, __ipipe_handle_irq or > > ipipe_trigger_irq so that the cascaded interrupts are pipelined. > > > > So, there are two possible fixes: > > - either fix the I-pipe patch so that the demux handler is invoked when > > the multiplexed interrupt is received for any domain, and triggers > > interrupts through ipipe_trigger_irq; > > Fixing the I-pipe is the way to go, definitely. If I understand this > correctly, the best way to handle the demux case is to implement an > I-pipe variant of the demultiplexing code currently available in > pxa_gpio_demux_handler(), which would be called from __ipipe_grab_irq() > when processing the multiplexed interrupt channel. > > The I-pipe-specific demux code would then log the decoded IRQ by a call > to __ipipe_handle_irq() instead of running the root handler directly > (i.e. desc_handle_irq), which would in turn preserve the ability to > "wire" GPIO interrupts and also handle IRQ stickiness issues. > > This would resemble what we are doing right now on the powerpc arch, > pulling the received IRQ number from a request to the machine-dependent > layer. I'd see no issue in duplicating such handling since this is a > rather short piece of invariant, hw-specific code which would not depend > on Linux internals, but rather on the I-pipe's. Ok, here is an implementation of this idea for integrator_cp, sa1100 and pxa as a patch that should be applied after the ipipe patch. As usual, Bart, could you test it ? -- Gilles Chanteperdrix. [-- Attachment #2: ipipe-sa1100-pxa.3.patch --] [-- Type: text/plain, Size: 16628 bytes --] diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/kernel/ipipe-root.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/kernel/ipipe-root.c 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c 2006-08-03 17:51:52.000000000 +0200 @@ -302,7 +302,10 @@ asmlinkage int __ipipe_grab_irq(int irq, } } - __ipipe_handle_irq(irq, regs); + if (__ipipe_mach_irq_mux_p(irq)) + __ipipe_mach_demux_irq(irq, regs); + else + __ipipe_handle_irq(irq, regs); ipipe_load_cpuid(); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-integrator/integrator_cp.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-integrator/integrator_cp.c 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c 2006-08-03 19:05:46.000000000 +0200 @@ -20,6 +20,7 @@ #include <linux/amba/bus.h> #include <linux/amba/kmi.h> #include <linux/amba/clcd.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/io.h> @@ -220,6 +221,31 @@ sic_handle_irq(unsigned int irq, struct } while (status); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + unsigned long status = sic_readl(INTCP_VA_SIC_BASE + IRQ_STATUS); + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + + if (status == 0) { + do_bad_IRQ(irq, desc_unused, regs); + return; + } + + do { + irq = ffs(status) - 1; + status &= ~(1 << irq); + + irq += IRQ_SIC_START; + + __ipipe_handle_irq(irq, regs); + } while (status); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + static void __init intcp_init_irq(void) { unsigned int i; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-03 19:04:52.000000000 +0200 @@ -16,6 +16,7 @@ #include <linux/module.h> #include <linux/interrupt.h> #include <linux/ptrace.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/irq.h> @@ -214,6 +215,77 @@ static void pxa_gpio_demux_handler(unsig } while (loop); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + unsigned int mask; + int loop; + + do { + loop = 0; + + mask = GEDR0 & ~3; + if (mask) { + GEDR0 = mask; + irq = IRQ_GPIO(2); + mask >>= 2; + do { + if (mask & 1) + __ipipe_handle_irq(irq, regs); + irq++; + mask >>= 1; + } while (mask); + loop = 1; + } + + mask = GEDR1; + if (mask) { + GEDR1 = mask; + irq = IRQ_GPIO(32); + do { + if (mask & 1) + __ipipe_handle_irq(irq, regs); + irq++; + mask >>= 1; + } while (mask); + loop = 1; + } + + mask = GEDR2; + if (mask) { + GEDR2 = mask; + irq = IRQ_GPIO(64); + do { + if (mask & 1) + __ipipe_handle_irq(irq, regs); + irq++; + mask >>= 1; + } while (mask); + loop = 1; + } + +#if PXA_LAST_GPIO >= 96 + mask = GEDR3; + if (mask) { + GEDR3 = mask; + irq = IRQ_GPIO(96); + do { + if (mask & 1) + __ipipe_handle_irq(irq, regs); + irq++; + mask >>= 1; + } while (mask); + loop = 1; + } +#endif + } while (loop); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + static void pxa_ack_muxed_gpio(unsigned int irq) { int gpio = irq - IRQ_GPIO(2) + 2; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c 2006-08-02 19:17:30.000000000 +0200 @@ -19,6 +19,7 @@ #include <linux/signal.h> #include <linux/errno.h> #include <linux/sched.h> +#include <linux/module.h> #include <asm/system.h> #include <asm/hardware.h> @@ -30,6 +31,23 @@ #include <asm/arch/pxa-regs.h> +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int pxa_timer_initialized; +static unsigned long pxa_jiffies; +#endif /* CONFIG_IPIPE */ + static inline unsigned long pxa_get_rtc_time(void) { return RCNR; @@ -54,6 +72,9 @@ static unsigned long pxa_gettimeoffset ( { long ticks_to_match, elapsed, usec; +#ifdef CONFIG_IPIPE + if (!__ipipe_mach_timerstolen) { +#endif /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; @@ -63,6 +84,10 @@ static unsigned long pxa_gettimeoffset ( /* don't get fooled by the workaround in pxa_timer_interrupt() */ if (elapsed <= 0) return 0; +#ifdef CONFIG_IPIPE + } else + elapsed = OSCR - pxa_jiffies * LATCH; +#endif /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; @@ -105,9 +130,27 @@ pxa_timer_interrupt(int irq, void *dev_i * affect things only when the timer IRQ has been delayed by nearly * exactly one tick period which should be a pretty rare event. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) { + timer_tick(regs); + ++pxa_jiffies; + } else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); +#ifdef CONFIG_IPIPE + ++pxa_jiffies; +#else /* !CONFIG_IPIPE */ OSSR = OSSR_M0; /* Clear match on timer 0 */ +#endif /* !CONFIG_IPIPE */ next_match = (OSMR0 += LATCH); } while( (signed long)(next_match - OSCR) <= 8 ); @@ -139,6 +182,10 @@ static void __init pxa_timer_init(void) setup_irq(IRQ_OST0, &pxa_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + pxa_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -216,3 +263,69 @@ struct sys_timer pxa_timer = { .dyn_tick = &pxa_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(pxa_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + if (delay > 8) { + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); + } else + ipipe_trigger_irq(IRQ_OST0); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/irq.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c 2006-08-03 19:05:03.000000000 +0200 @@ -14,6 +14,7 @@ #include <linux/ioport.h> #include <linux/ptrace.h> #include <linux/sysdev.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/irq.h> @@ -136,6 +137,37 @@ sa1100_high_gpio_handler(unsigned int ir } while (mask); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + unsigned int mask; + + mask = GEDR & 0xfffff800; + do { + /* + * clear down all currently active IRQ sources. + * We will be processing them all. + */ + GEDR = mask; + + irq = IRQ_GPIO11; + mask >>= 11; + do { + if (mask & 1) + __ipipe_handle_irq(irq, regs); + mask >>= 1; + irq++; + } while (mask); + + mask = GEDR & 0xfffff800; + } while (mask); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + /* * Like GPIO0 to 10, GPIO11-27 IRQs need to be handled specially. * In addition, the IRQs are all collected up into one bit in the diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c 2006-08-02 19:09:06.000000000 +0200 @@ -13,6 +13,7 @@ #include <linux/interrupt.h> #include <linux/timex.h> #include <linux/signal.h> +#include <linux/module.h> #include <asm/mach/time.h> #include <asm/hardware.h> @@ -20,6 +21,23 @@ #define RTC_DEF_DIVIDER (32768 - 1) #define RTC_DEF_TRIM 0 +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int sa1100_timer_initialized; +static unsigned long sa1100_jiffies; +#endif /* CONFIG_IPIPE */ + static unsigned long __init sa1100_get_rtc_time(void) { /* @@ -58,11 +76,18 @@ static unsigned long sa1100_gettimeoffse { unsigned long ticks_to_match, elapsed, usec; +#ifdef CONFIG_IPIPE + if (!__ipipe_mach_timerstolen) { +#endif /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; /* We need elapsed ticks since last match */ elapsed = LATCH - ticks_to_match; +#ifdef CONFIG_IPIPE + } else + elapsed = OSCR - sa1100_jiffies * LATCH; +#endif /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; @@ -97,9 +122,27 @@ sa1100_timer_interrupt(int irq, void *de * ensured, hence we can use do_gettimeofday() from interrupt * handlers. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) { + timer_tick(regs); + ++sa1100_jiffies; + } else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); +#ifdef CONFIG_IPIPE + ++sa1100_jiffies; +#else /* !CONFIG_IPIPE */ OSSR = OSSR_M0; /* Clear match on timer 0 */ +#endif /* !CONFIG_IPIPE */ next_match = (OSMR0 += LATCH); } while ((signed long)(next_match - OSCR) <= 0); @@ -131,6 +174,10 @@ static void __init sa1100_timer_init(voi setup_irq(IRQ_OST0, &sa1100_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + sa1100_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -209,3 +256,66 @@ struct sys_timer sa1100_timer = { .dyn_tick = &sa1100_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(sa1100_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-integrator/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-integrator/irqs.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/irqs.h 2006-08-03 18:17:55.000000000 +0200 @@ -80,3 +80,6 @@ #define NR_IRQS 47 +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_CP_CPPLDINT) +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-pxa/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-pxa/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-pxa/irqs.h 2006-07-15 20:06:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-pxa/irqs.h 2006-08-03 18:01:15.000000000 +0200 @@ -73,6 +73,10 @@ ((i) - IRQ_GPIO(2) + 2) #define IRQ_TO_GPIO(i) (((i) < IRQ_GPIO(2)) ? ((i) - IRQ_GPIO0) : IRQ_TO_GPIO_2_x(i)) +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_GPIO_2_x) +#endif /* CONFIG_IPIPE */ + #if defined(CONFIG_PXA25x) #define PXA_LAST_GPIO 80 #elif defined(CONFIG_PXA27x) diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-sa1100/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-sa1100/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-sa1100/irqs.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-sa1100/irqs.h 2006-08-03 18:14:34.000000000 +0200 @@ -145,6 +145,10 @@ #define IRQ_LOCOMO_SPI_OVRN (IRQ_BOARD_END + 20) #define IRQ_LOCOMO_SPI_TEND (IRQ_BOARD_END + 21) +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_GPIO11_27) +#endif /* CONFIG_IPIPE */ + /* * Figure out the MAX IRQ number. * diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/ipipe.h 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h 2006-08-03 17:52:02.000000000 +0200 @@ -117,6 +117,7 @@ extern void __ipipe_mach_acktimer(void); extern unsigned long long __ipipe_mach_get_tsc(void); extern void __ipipe_mach_set_dec(unsigned long); extern unsigned long __ipipe_mach_get_dec(void); +extern void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs); #define ipipe_read_tsc(t) do { t = __ipipe_mach_get_tsc(); } while (0) #define __ipipe_read_timebase() __ipipe_mach_get_tsc() ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-03 17:14 ` Gilles Chanteperdrix @ 2006-08-04 6:10 ` Bart Jonkers 2006-08-04 6:10 ` Detlef Vollmann 2006-08-04 14:52 ` Bart Jonkers 2 siblings, 0 replies; 36+ messages in thread From: Bart Jonkers @ 2006-08-04 6:10 UTC (permalink / raw) To: xenomai On Thu, 2006-08-03 at 19:14 +0200, Gilles Chanteperdrix wrote: > Philippe Gerum wrote: > > On Thu, 2006-08-03 at 15:18 +0200, Gilles Chanteperdrix wrote: > > > Bart Jonkers wrote: > > > > > Here is a new version of the ipipe-sa1100-pxa patch that unmaks > > > > > interrupts at the end of the demux handlers, and that attempt to fix the > > > > > gettimeoffset issue. I have run 20 minutes (time for OSCR to wrap) latency > > > > > with a test program verifying that time returned by gettimeofday is > > > > > never going backward. > > > > > > > > > > The patch and the test program are attached, it would be nice if you > > > > > could test them. > > > > I have tested the patch and ran the test program. > > > > The test program give no problems. > > > > > > > > The interrupt problem in Linux is also solved. But I have another one. > > > > I have created a small real-time kernel module which uses the native > > > > interface to handle an GPIO interrupt. The problem is that Xenomai > > > > doesn't see the interrupt. When I use GPIO0 (which doesn't use the > > > > chained handler) to receive the interrupt everything works as it should. > > > > > > > > So it seems to be that xenomai have a problem with the chained handler. > > > > Does someone have an idea to solve this problem? > > > > > > The problem is that the demux handler is a root domain handler, and > > > directly call the cascaded interrupts root domain handlers through > > > desc_handle_irq, whereas in order for the I-pipe to intercept the > > > interrupt for any domain, the demux handler should be run for any > > > domain, and invoke __ipipe_grab_irq, __ipipe_handle_irq or > > > ipipe_trigger_irq so that the cascaded interrupts are pipelined. > > > > > > So, there are two possible fixes: > > > - either fix the I-pipe patch so that the demux handler is invoked when > > > the multiplexed interrupt is received for any domain, and triggers > > > interrupts through ipipe_trigger_irq; > > > > Fixing the I-pipe is the way to go, definitely. If I understand this > > correctly, the best way to handle the demux case is to implement an > > I-pipe variant of the demultiplexing code currently available in > > pxa_gpio_demux_handler(), which would be called from __ipipe_grab_irq() > > when processing the multiplexed interrupt channel. > > > > The I-pipe-specific demux code would then log the decoded IRQ by a call > > to __ipipe_handle_irq() instead of running the root handler directly > > (i.e. desc_handle_irq), which would in turn preserve the ability to > > "wire" GPIO interrupts and also handle IRQ stickiness issues. > > > > This would resemble what we are doing right now on the powerpc arch, > > pulling the received IRQ number from a request to the machine-dependent > > layer. I'd see no issue in duplicating such handling since this is a > > rather short piece of invariant, hw-specific code which would not depend > > on Linux internals, but rather on the I-pipe's. > > Ok, here is an implementation of this idea for integrator_cp, sa1100 and > pxa as a patch that should be applied after the ipipe patch. As usual, > Bart, could you test it ? Tested. It works, now it is possible to receive GPIO interrupts in the real-time domain. Bart > > plain text document attachment (ipipe-sa1100-pxa.3.patch) > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/kernel/ipipe-root.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c > --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/kernel/ipipe-root.c 2006-08-03 18:08:48.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c 2006-08-03 17:51:52.000000000 +0200 > @@ -302,7 +302,10 @@ asmlinkage int __ipipe_grab_irq(int irq, > } > } > > - __ipipe_handle_irq(irq, regs); > + if (__ipipe_mach_irq_mux_p(irq)) > + __ipipe_mach_demux_irq(irq, regs); > + else > + __ipipe_handle_irq(irq, regs); > > ipipe_load_cpuid(); > > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-integrator/integrator_cp.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c > --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-integrator/integrator_cp.c 2006-08-03 18:08:48.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c 2006-08-03 19:05:46.000000000 +0200 > @@ -20,6 +20,7 @@ > #include <linux/amba/bus.h> > #include <linux/amba/kmi.h> > #include <linux/amba/clcd.h> > +#include <linux/ipipe.h> > > #include <asm/hardware.h> > #include <asm/io.h> > @@ -220,6 +221,31 @@ sic_handle_irq(unsigned int irq, struct > } while (status); > } > > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) > +{ > + unsigned long status = sic_readl(INTCP_VA_SIC_BASE + IRQ_STATUS); > + struct irqdesc *desc_unused = irq_desc + irq; > + unsigned irq_unused = irq; > + > + if (status == 0) { > + do_bad_IRQ(irq, desc_unused, regs); > + return; > + } > + > + do { > + irq = ffs(status) - 1; > + status &= ~(1 << irq); > + > + irq += IRQ_SIC_START; > + > + __ipipe_handle_irq(irq, regs); > + } while (status); > + > + desc_unused->chip->unmask(irq_unused); > +} > +#endif /* CONFIG_IPIPE */ > + > static void __init intcp_init_irq(void) > { > unsigned int i; > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c > --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-03 19:04:52.000000000 +0200 > @@ -16,6 +16,7 @@ > #include <linux/module.h> > #include <linux/interrupt.h> > #include <linux/ptrace.h> > +#include <linux/ipipe.h> > > #include <asm/hardware.h> > #include <asm/irq.h> > @@ -214,6 +215,77 @@ static void pxa_gpio_demux_handler(unsig > } while (loop); > } > > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) > +{ > + struct irqdesc *desc_unused = irq_desc + irq; > + unsigned irq_unused = irq; > + unsigned int mask; > + int loop; > + > + do { > + loop = 0; > + > + mask = GEDR0 & ~3; > + if (mask) { > + GEDR0 = mask; > + irq = IRQ_GPIO(2); > + mask >>= 2; > + do { > + if (mask & 1) > + __ipipe_handle_irq(irq, regs); > + irq++; > + mask >>= 1; > + } while (mask); > + loop = 1; > + } > + > + mask = GEDR1; > + if (mask) { > + GEDR1 = mask; > + irq = IRQ_GPIO(32); > + do { > + if (mask & 1) > + __ipipe_handle_irq(irq, regs); > + irq++; > + mask >>= 1; > + } while (mask); > + loop = 1; > + } > + > + mask = GEDR2; > + if (mask) { > + GEDR2 = mask; > + irq = IRQ_GPIO(64); > + do { > + if (mask & 1) > + __ipipe_handle_irq(irq, regs); > + irq++; > + mask >>= 1; > + } while (mask); > + loop = 1; > + } > + > +#if PXA_LAST_GPIO >= 96 > + mask = GEDR3; > + if (mask) { > + GEDR3 = mask; > + irq = IRQ_GPIO(96); > + do { > + if (mask & 1) > + __ipipe_handle_irq(irq, regs); > + irq++; > + mask >>= 1; > + } while (mask); > + loop = 1; > + } > +#endif > + } while (loop); > + > + desc_unused->chip->unmask(irq_unused); > +} > +#endif /* CONFIG_IPIPE */ > + > static void pxa_ack_muxed_gpio(unsigned int irq) > { > int gpio = irq - IRQ_GPIO(2) + 2; > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c > --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/time.c 2006-05-07 15:36:35.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c 2006-08-02 19:17:30.000000000 +0200 > @@ -19,6 +19,7 @@ > #include <linux/signal.h> > #include <linux/errno.h> > #include <linux/sched.h> > +#include <linux/module.h> > > #include <asm/system.h> > #include <asm/hardware.h> > @@ -30,6 +31,23 @@ > #include <asm/arch/pxa-regs.h> > > > +#ifdef CONFIG_IPIPE > +#ifdef CONFIG_NO_IDLE_HZ > +#error "dynamic tick timer not yet supported with IPIPE" > +#endif /* CONFIG_NO_IDLE_HZ */ > +int __ipipe_mach_timerint = IRQ_OST0; > +EXPORT_SYMBOL(__ipipe_mach_timerint); > + > +int __ipipe_mach_timerstolen = 0; > +EXPORT_SYMBOL(__ipipe_mach_timerstolen); > + > +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; > +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); > + > +static int pxa_timer_initialized; > +static unsigned long pxa_jiffies; > +#endif /* CONFIG_IPIPE */ > + > static inline unsigned long pxa_get_rtc_time(void) > { > return RCNR; > @@ -54,6 +72,9 @@ static unsigned long pxa_gettimeoffset ( > { > long ticks_to_match, elapsed, usec; > > +#ifdef CONFIG_IPIPE > + if (!__ipipe_mach_timerstolen) { > +#endif > /* Get ticks before next timer match */ > ticks_to_match = OSMR0 - OSCR; > > @@ -63,6 +84,10 @@ static unsigned long pxa_gettimeoffset ( > /* don't get fooled by the workaround in pxa_timer_interrupt() */ > if (elapsed <= 0) > return 0; > +#ifdef CONFIG_IPIPE > + } else > + elapsed = OSCR - pxa_jiffies * LATCH; > +#endif > > /* Now convert them to usec */ > usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; > @@ -105,9 +130,27 @@ pxa_timer_interrupt(int irq, void *dev_i > * affect things only when the timer IRQ has been delayed by nearly > * exactly one tick period which should be a pretty rare event. > */ > +#ifdef CONFIG_IPIPE > + /* > + * - if Linux is running natively (no ipipe), ack and reprogram the timer > + * - if Linux is running under ipipe, but it still has the control over > + * the timer (no Xenomai for example), then reprogram the timer (ipipe > + * has already acked it) > + * - if some other domain has taken over the timer, then do nothing > + * (ipipe has acked it, and the other domain has reprogramed it) > + */ > + if (__ipipe_mach_timerstolen) { > + timer_tick(regs); > + ++pxa_jiffies; > + } else > +#endif /* CONFIG_IPIPE */ > do { > timer_tick(regs); > +#ifdef CONFIG_IPIPE > + ++pxa_jiffies; > +#else /* !CONFIG_IPIPE */ > OSSR = OSSR_M0; /* Clear match on timer 0 */ > +#endif /* !CONFIG_IPIPE */ > next_match = (OSMR0 += LATCH); > } while( (signed long)(next_match - OSCR) <= 8 ); > > @@ -139,6 +182,10 @@ static void __init pxa_timer_init(void) > setup_irq(IRQ_OST0, &pxa_timer_irq); > OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ > OSCR = 0; /* initialize free-running timer */ > + > +#ifdef CONFIG_IPIPE > + pxa_timer_initialized = 1; > +#endif /* CONFIG_IPIPE */ > } > > #ifdef CONFIG_NO_IDLE_HZ > @@ -216,3 +263,69 @@ struct sys_timer pxa_timer = { > .dyn_tick = &pxa_dyn_tick, > #endif > }; > + > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_acktimer(void) > +{ > + OSSR = OSSR_M0; /* Clear match on timer 0 */ > +} > + > +unsigned long long __ipipe_mach_get_tsc(void) > +{ > + if (likely(pxa_timer_initialized)) { > + static union { > +#ifdef __BIG_ENDIAN > + struct { > + unsigned long high; > + unsigned long low; > + }; > +#else /* __LITTLE_ENDIAN */ > + struct { > + unsigned long low; > + unsigned long high; > + }; > +#endif /* __LITTLE_ENDIAN */ > + unsigned long long full; > + } tsc[NR_CPUS], *local_tsc; > + unsigned long stamp, flags; > + unsigned long long result; > + > + local_irq_save_hw(flags); > + local_tsc = &tsc[ipipe_processor_id()]; > + stamp = OSCR; > + if (unlikely(stamp < local_tsc->low)) > + /* 32 bit counter wrapped, increment high word. */ > + local_tsc->high++; > + local_tsc->low = stamp; > + result = local_tsc->full; > + local_irq_restore_hw(flags); > + > + return result; > + } > + > + return 0; > +} > +EXPORT_SYMBOL(__ipipe_mach_get_tsc); > + > +/* > + * Reprogram the timer > + */ > + > +void __ipipe_mach_set_dec(unsigned long delay) > +{ > + if (delay > 8) { > + unsigned long flags; > + > + local_irq_save_hw(flags); > + OSMR0 = delay + OSCR; > + local_irq_restore_hw(flags); > + } else > + ipipe_trigger_irq(IRQ_OST0); > +} > +EXPORT_SYMBOL(__ipipe_mach_set_dec); > + > +unsigned long __ipipe_mach_get_dec(void) > +{ > + return OSMR0 - OSCR; > +} > +#endif /* CONFIG_IPIPE */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c > --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/irq.c 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c 2006-08-03 19:05:03.000000000 +0200 > @@ -14,6 +14,7 @@ > #include <linux/ioport.h> > #include <linux/ptrace.h> > #include <linux/sysdev.h> > +#include <linux/ipipe.h> > > #include <asm/hardware.h> > #include <asm/irq.h> > @@ -136,6 +137,37 @@ sa1100_high_gpio_handler(unsigned int ir > } while (mask); > } > > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) > +{ > + struct irqdesc *desc_unused = irq_desc + irq; > + unsigned irq_unused = irq; > + unsigned int mask; > + > + mask = GEDR & 0xfffff800; > + do { > + /* > + * clear down all currently active IRQ sources. > + * We will be processing them all. > + */ > + GEDR = mask; > + > + irq = IRQ_GPIO11; > + mask >>= 11; > + do { > + if (mask & 1) > + __ipipe_handle_irq(irq, regs); > + mask >>= 1; > + irq++; > + } while (mask); > + > + mask = GEDR & 0xfffff800; > + } while (mask); > + > + desc_unused->chip->unmask(irq_unused); > +} > +#endif /* CONFIG_IPIPE */ > + > /* > * Like GPIO0 to 10, GPIO11-27 IRQs need to be handled specially. > * In addition, the IRQs are all collected up into one bit in the > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c > --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/time.c 2006-05-07 15:36:35.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c 2006-08-02 19:09:06.000000000 +0200 > @@ -13,6 +13,7 @@ > #include <linux/interrupt.h> > #include <linux/timex.h> > #include <linux/signal.h> > +#include <linux/module.h> > > #include <asm/mach/time.h> > #include <asm/hardware.h> > @@ -20,6 +21,23 @@ > #define RTC_DEF_DIVIDER (32768 - 1) > #define RTC_DEF_TRIM 0 > > +#ifdef CONFIG_IPIPE > +#ifdef CONFIG_NO_IDLE_HZ > +#error "dynamic tick timer not yet supported with IPIPE" > +#endif /* CONFIG_NO_IDLE_HZ */ > +int __ipipe_mach_timerint = IRQ_OST0; > +EXPORT_SYMBOL(__ipipe_mach_timerint); > + > +int __ipipe_mach_timerstolen = 0; > +EXPORT_SYMBOL(__ipipe_mach_timerstolen); > + > +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; > +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); > + > +static int sa1100_timer_initialized; > +static unsigned long sa1100_jiffies; > +#endif /* CONFIG_IPIPE */ > + > static unsigned long __init sa1100_get_rtc_time(void) > { > /* > @@ -58,11 +76,18 @@ static unsigned long sa1100_gettimeoffse > { > unsigned long ticks_to_match, elapsed, usec; > > +#ifdef CONFIG_IPIPE > + if (!__ipipe_mach_timerstolen) { > +#endif > /* Get ticks before next timer match */ > ticks_to_match = OSMR0 - OSCR; > > /* We need elapsed ticks since last match */ > elapsed = LATCH - ticks_to_match; > +#ifdef CONFIG_IPIPE > + } else > + elapsed = OSCR - sa1100_jiffies * LATCH; > +#endif > > /* Now convert them to usec */ > usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; > @@ -97,9 +122,27 @@ sa1100_timer_interrupt(int irq, void *de > * ensured, hence we can use do_gettimeofday() from interrupt > * handlers. > */ > +#ifdef CONFIG_IPIPE > + /* > + * - if Linux is running natively (no ipipe), ack and reprogram the timer > + * - if Linux is running under ipipe, but it still has the control over > + * the timer (no Xenomai for example), then reprogram the timer (ipipe > + * has already acked it) > + * - if some other domain has taken over the timer, then do nothing > + * (ipipe has acked it, and the other domain has reprogramed it) > + */ > + if (__ipipe_mach_timerstolen) { > + timer_tick(regs); > + ++sa1100_jiffies; > + } else > +#endif /* CONFIG_IPIPE */ > do { > timer_tick(regs); > +#ifdef CONFIG_IPIPE > + ++sa1100_jiffies; > +#else /* !CONFIG_IPIPE */ > OSSR = OSSR_M0; /* Clear match on timer 0 */ > +#endif /* !CONFIG_IPIPE */ > next_match = (OSMR0 += LATCH); > } while ((signed long)(next_match - OSCR) <= 0); > > @@ -131,6 +174,10 @@ static void __init sa1100_timer_init(voi > setup_irq(IRQ_OST0, &sa1100_timer_irq); > OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ > OSCR = 0; /* initialize free-running timer */ > + > +#ifdef CONFIG_IPIPE > + sa1100_timer_initialized = 1; > +#endif /* CONFIG_IPIPE */ > } > > #ifdef CONFIG_NO_IDLE_HZ > @@ -209,3 +256,66 @@ struct sys_timer sa1100_timer = { > .dyn_tick = &sa1100_dyn_tick, > #endif > }; > + > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_acktimer(void) > +{ > + OSSR = OSSR_M0; /* Clear match on timer 0 */ > +} > + > +unsigned long long __ipipe_mach_get_tsc(void) > +{ > + if (likely(sa1100_timer_initialized)) { > + static union { > +#ifdef __BIG_ENDIAN > + struct { > + unsigned long high; > + unsigned long low; > + }; > +#else /* __LITTLE_ENDIAN */ > + struct { > + unsigned long low; > + unsigned long high; > + }; > +#endif /* __LITTLE_ENDIAN */ > + unsigned long long full; > + } tsc[NR_CPUS], *local_tsc; > + unsigned long stamp, flags; > + unsigned long long result; > + > + local_irq_save_hw(flags); > + local_tsc = &tsc[ipipe_processor_id()]; > + stamp = OSCR; > + if (unlikely(stamp < local_tsc->low)) > + /* 32 bit counter wrapped, increment high word. */ > + local_tsc->high++; > + local_tsc->low = stamp; > + result = local_tsc->full; > + local_irq_restore_hw(flags); > + > + return result; > + } > + > + return 0; > +} > +EXPORT_SYMBOL(__ipipe_mach_get_tsc); > + > +/* > + * Reprogram the timer > + */ > + > +void __ipipe_mach_set_dec(unsigned long delay) > +{ > + unsigned long flags; > + > + local_irq_save_hw(flags); > + OSMR0 = delay + OSCR; > + local_irq_restore_hw(flags); > +} > +EXPORT_SYMBOL(__ipipe_mach_set_dec); > + > +unsigned long __ipipe_mach_get_dec(void) > +{ > + return OSMR0 - OSCR; > +} > +#endif /* CONFIG_IPIPE */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-integrator/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/irqs.h > --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-integrator/irqs.h 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/irqs.h 2006-08-03 18:17:55.000000000 +0200 > @@ -80,3 +80,6 @@ > > #define NR_IRQS 47 > > +#ifdef CONFIG_IPIPE > +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_CP_CPPLDINT) > +#endif /* CONFIG_IPIPE */ > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-pxa/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-pxa/irqs.h > --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-pxa/irqs.h 2006-07-15 20:06:03.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-pxa/irqs.h 2006-08-03 18:01:15.000000000 +0200 > @@ -73,6 +73,10 @@ > ((i) - IRQ_GPIO(2) + 2) > #define IRQ_TO_GPIO(i) (((i) < IRQ_GPIO(2)) ? ((i) - IRQ_GPIO0) : IRQ_TO_GPIO_2_x(i)) > > +#ifdef CONFIG_IPIPE > +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_GPIO_2_x) > +#endif /* CONFIG_IPIPE */ > + > #if defined(CONFIG_PXA25x) > #define PXA_LAST_GPIO 80 > #elif defined(CONFIG_PXA27x) > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-sa1100/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-sa1100/irqs.h > --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-sa1100/irqs.h 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-sa1100/irqs.h 2006-08-03 18:14:34.000000000 +0200 > @@ -145,6 +145,10 @@ > #define IRQ_LOCOMO_SPI_OVRN (IRQ_BOARD_END + 20) > #define IRQ_LOCOMO_SPI_TEND (IRQ_BOARD_END + 21) > > +#ifdef CONFIG_IPIPE > +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_GPIO11_27) > +#endif /* CONFIG_IPIPE */ > + > /* > * Figure out the MAX IRQ number. > * > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h > --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/ipipe.h 2006-08-03 18:08:48.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h 2006-08-03 17:52:02.000000000 +0200 > @@ -117,6 +117,7 @@ extern void __ipipe_mach_acktimer(void); > extern unsigned long long __ipipe_mach_get_tsc(void); > extern void __ipipe_mach_set_dec(unsigned long); > extern unsigned long __ipipe_mach_get_dec(void); > +extern void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs); > > #define ipipe_read_tsc(t) do { t = __ipipe_mach_get_tsc(); } while (0) > #define __ipipe_read_timebase() __ipipe_mach_get_tsc() ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-03 17:14 ` Gilles Chanteperdrix 2006-08-04 6:10 ` Bart Jonkers @ 2006-08-04 6:10 ` Detlef Vollmann 2006-08-04 9:04 ` Philippe Gerum 2006-08-04 11:37 ` Gilles Chanteperdrix 2006-08-04 14:52 ` Bart Jonkers 2 siblings, 2 replies; 36+ messages in thread From: Detlef Vollmann @ 2006-08-04 6:10 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai Gilles Chanteperdrix wrote: > Philippe Gerum wrote: > > Fixing the I-pipe is the way to go, definitely. If I understand this > > correctly, the best way to handle the demux case is to implement an > > I-pipe variant of the demultiplexing code currently available in > > pxa_gpio_demux_handler(), which would be called from __ipipe_grab_irq() > > when processing the multiplexed interrupt channel. > > > > The I-pipe-specific demux code would then log the decoded IRQ by a call > > to __ipipe_handle_irq() instead of running the root handler directly > > (i.e. desc_handle_irq), which would in turn preserve the ability to > > "wire" GPIO interrupts and also handle IRQ stickiness issues. This was also my first idea. But after a bit of musing on that idea, I'm not really sure that this is the best way to go. We're talking here about more than 100 bits that have to be tested one after the other (on PXA270, it's 116, though the actual code checks 126 bits), and even on a fast CPU this takes some time. So I've thought about a way to register a GPIO-IRQ into I-pipe that gets checked directly under I-pipe, while all other GPIOs are only checked and handled in the domain that handles the original multiplexing IRQ. But I have to admit that I have no idea how to implement that... [snip] > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-03 19:04:52.000000000 +0200 > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) > + __ipipe_handle_irq(irq, regs); Just to check: this only marks the interrupt, but not actually handles it, doesn't it? Detlef -- Detlef Vollmann vollmann engineering gmbh Linux and C++ for Embedded Systems http://www.vollmann.ch/ Linux for PXA270 Colibri module: http://www.vollmann.ch/en/colibri/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-04 6:10 ` Detlef Vollmann @ 2006-08-04 9:04 ` Philippe Gerum 2006-08-04 11:37 ` Gilles Chanteperdrix 1 sibling, 0 replies; 36+ messages in thread From: Philippe Gerum @ 2006-08-04 9:04 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai On Fri, 2006-08-04 at 08:10 +0200, Detlef Vollmann wrote: > Gilles Chanteperdrix wrote: > > Philippe Gerum wrote: > > > Fixing the I-pipe is the way to go, definitely. If I understand this > > > correctly, the best way to handle the demux case is to implement an > > > I-pipe variant of the demultiplexing code currently available in > > > pxa_gpio_demux_handler(), which would be called from __ipipe_grab_irq() > > > when processing the multiplexed interrupt channel. > > > > > > The I-pipe-specific demux code would then log the decoded IRQ by a call > > > to __ipipe_handle_irq() instead of running the root handler directly > > > (i.e. desc_handle_irq), which would in turn preserve the ability to > > > "wire" GPIO interrupts and also handle IRQ stickiness issues. > This was also my first idea. But after a bit of musing on that > idea, I'm not really sure that this is the best way to go. > We're talking here about more than 100 bits that have to be tested > one after the other (on PXA270, it's 116, though the actual code > checks 126 bits), and even on a fast CPU this takes some time. > > So I've thought about a way to register a GPIO-IRQ into I-pipe > that gets checked directly under I-pipe, while all other GPIOs > are only checked and handled in the domain that handles the > original multiplexing IRQ. > But I have to admit that I have no idea how to implement that... > Provided that every IRQ notification goes through __ipipe_grab_irq, including the GPIO_2_x one, then you just need to check for irq == GPIO_2_x in the latter routine and proceed to the demultiplexing only in this case. No? > [snip] > > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-03 19:04:52.000000000 +0200 > > > +#ifdef CONFIG_IPIPE > > +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) > > > + __ipipe_handle_irq(irq, regs); > Just to check: this only marks the interrupt, but not actually > handles it, doesn't it? > > Detlef > -- Philippe. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-04 6:10 ` Detlef Vollmann 2006-08-04 9:04 ` Philippe Gerum @ 2006-08-04 11:37 ` Gilles Chanteperdrix 1 sibling, 0 replies; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-08-04 11:37 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai Detlef Vollmann wrote: > Gilles Chanteperdrix wrote: > > Philippe Gerum wrote: > > > Fixing the I-pipe is the way to go, definitely. If I understand this > > > correctly, the best way to handle the demux case is to implement an > > > I-pipe variant of the demultiplexing code currently available in > > > pxa_gpio_demux_handler(), which would be called from __ipipe_grab_irq() > > > when processing the multiplexed interrupt channel. > > > > > > The I-pipe-specific demux code would then log the decoded IRQ by a call > > > to __ipipe_handle_irq() instead of running the root handler directly > > > (i.e. desc_handle_irq), which would in turn preserve the ability to > > > "wire" GPIO interrupts and also handle IRQ stickiness issues. > This was also my first idea. But after a bit of musing on that > idea, I'm not really sure that this is the best way to go. > We're talking here about more than 100 bits that have to be tested > one after the other (on PXA270, it's 116, though the actual code > checks 126 bits), and even on a fast CPU this takes some time. > > So I've thought about a way to register a GPIO-IRQ into I-pipe > that gets checked directly under I-pipe, while all other GPIOs > are only checked and handled in the domain that handles the > original multiplexing IRQ. > But I have to admit that I have no idea how to implement that... Maybe a reasonable alternative would be to use fls in order to find the set bits in the mask ? This way, if the masks are sparse, which they should be, we will skip a lot of checks. > > [snip] > > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-03 19:04:52.000000000 +0200 > > > +#ifdef CONFIG_IPIPE > > +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) > > > + __ipipe_handle_irq(irq, regs); > Just to check: this only marks the interrupt, but not actually > handles it, doesn't it? This handles it if the current or higher domains have a registered handler and are not stalled. -- Gilles Chanteperdrix. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-03 17:14 ` Gilles Chanteperdrix 2006-08-04 6:10 ` Bart Jonkers 2006-08-04 6:10 ` Detlef Vollmann @ 2006-08-04 14:52 ` Bart Jonkers 2006-08-13 13:16 ` Gilles Chanteperdrix 2 siblings, 1 reply; 36+ messages in thread From: Bart Jonkers @ 2006-08-04 14:52 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai On Thu, 2006-08-03 at 19:14 +0200, Gilles Chanteperdrix wrote: > Philippe Gerum wrote: > > On Thu, 2006-08-03 at 15:18 +0200, Gilles Chanteperdrix wrote: > > > Bart Jonkers wrote: > > > > > Here is a new version of the ipipe-sa1100-pxa patch that unmaks > > > > > interrupts at the end of the demux handlers, and that attempt to fix the > > > > > gettimeoffset issue. I have run 20 minutes (time for OSCR to wrap) latency > > > > > with a test program verifying that time returned by gettimeofday is > > > > > never going backward. > > > > > > > > > > The patch and the test program are attached, it would be nice if you > > > > > could test them. > > > > I have tested the patch and ran the test program. > > > > The test program give no problems. > > > > > > > > The interrupt problem in Linux is also solved. But I have another one. > > > > I have created a small real-time kernel module which uses the native > > > > interface to handle an GPIO interrupt. The problem is that Xenomai > > > > doesn't see the interrupt. When I use GPIO0 (which doesn't use the > > > > chained handler) to receive the interrupt everything works as it should. > > > > > > > > So it seems to be that xenomai have a problem with the chained handler. > > > > Does someone have an idea to solve this problem? > > > > > > The problem is that the demux handler is a root domain handler, and > > > directly call the cascaded interrupts root domain handlers through > > > desc_handle_irq, whereas in order for the I-pipe to intercept the > > > interrupt for any domain, the demux handler should be run for any > > > domain, and invoke __ipipe_grab_irq, __ipipe_handle_irq or > > > ipipe_trigger_irq so that the cascaded interrupts are pipelined. > > > > > > So, there are two possible fixes: > > > - either fix the I-pipe patch so that the demux handler is invoked when > > > the multiplexed interrupt is received for any domain, and triggers > > > interrupts through ipipe_trigger_irq; > > > > Fixing the I-pipe is the way to go, definitely. If I understand this > > correctly, the best way to handle the demux case is to implement an > > I-pipe variant of the demultiplexing code currently available in > > pxa_gpio_demux_handler(), which would be called from __ipipe_grab_irq() > > when processing the multiplexed interrupt channel. > > > > The I-pipe-specific demux code would then log the decoded IRQ by a call > > to __ipipe_handle_irq() instead of running the root handler directly > > (i.e. desc_handle_irq), which would in turn preserve the ability to > > "wire" GPIO interrupts and also handle IRQ stickiness issues. > > > > This would resemble what we are doing right now on the powerpc arch, > > pulling the received IRQ number from a request to the machine-dependent > > layer. I'd see no issue in duplicating such handling since this is a > > rather short piece of invariant, hw-specific code which would not depend > > on Linux internals, but rather on the I-pipe's. > > Ok, here is an implementation of this idea for integrator_cp, sa1100 and > pxa as a patch that should be applied after the ipipe patch. As usual, > Bart, could you test it ? > I'm testing the interrupt system a little bit more and have encountered a problem. I use a OS timer match register of the PXA to generate interrupts on a regular base. In the interrupt handler I reprogram the match register so the interrupt fires hole the time. In the interrupt handler happens nothing special (no special calls or something, just toggling of GPIO). When the interrupt stops (after 1000 times) everything still hangs. When I does this in the xenomai domain (native skin) Linux get stuck. User-space doesn't react anymore and some parts of the kernel also doesn't react. Pinging the system still works but a timer that toggles a LED doesn't do anything . When I does the same in the linux domain everything still works. Has somebody an idea? Bart ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-04 14:52 ` Bart Jonkers @ 2006-08-13 13:16 ` Gilles Chanteperdrix 2006-09-10 8:56 ` Detlef Vollmann 0 siblings, 1 reply; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-08-13 13:16 UTC (permalink / raw) To: Bart Jonkers; +Cc: xenomai [-- Attachment #1: message body and .signature --] [-- Type: text/plain, Size: 1279 bytes --] Bart Jonkers wrote: > I'm testing the interrupt system a little bit more and have encountered > a problem. > I use a OS timer match register of the PXA to generate interrupts on a > regular base. In the interrupt handler I reprogram the match register so > the interrupt fires hole the time. In the interrupt handler happens > nothing special (no special calls or something, just toggling of GPIO). > When the interrupt stops (after 1000 times) everything still hangs. > > When I does this in the xenomai domain (native skin) Linux get stuck. > User-space doesn't react anymore and some parts of the kernel also > doesn't react. Pinging the system still works but a timer that toggles a > LED doesn't do anything . > > When I does the same in the linux domain everything still works. > > Has somebody an idea? Attached a new version of the patch (including the patch for the __xchg operation, so that this patch can be applied incrementally after the official I-pipe patch) which should be more greedy when demultiplexing the GPIO interrupt. As for your issue, it is difficult to say anything without seeing your code, but since you say that the interrupt fires "all the time", are you sure you are not starving Linux ? -- Gilles Chanteperdrix. [-- Attachment #2: ipipe-sa1100-pxa.5.patch --] [-- Type: text/plain, Size: 17508 bytes --] diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/kernel/ipipe-root.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/kernel/ipipe-root.c 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c 2006-08-03 17:51:52.000000000 +0200 @@ -302,7 +302,10 @@ asmlinkage int __ipipe_grab_irq(int irq, } } - __ipipe_handle_irq(irq, regs); + if (__ipipe_mach_irq_mux_p(irq)) + __ipipe_mach_demux_irq(irq, regs); + else + __ipipe_handle_irq(irq, regs); ipipe_load_cpuid(); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-integrator/integrator_cp.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-integrator/integrator_cp.c 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c 2006-08-03 19:05:46.000000000 +0200 @@ -20,6 +20,7 @@ #include <linux/amba/bus.h> #include <linux/amba/kmi.h> #include <linux/amba/clcd.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/io.h> @@ -220,6 +221,31 @@ sic_handle_irq(unsigned int irq, struct } while (status); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + unsigned long status = sic_readl(INTCP_VA_SIC_BASE + IRQ_STATUS); + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + + if (status == 0) { + do_bad_IRQ(irq, desc_unused, regs); + return; + } + + do { + irq = ffs(status) - 1; + status &= ~(1 << irq); + + irq += IRQ_SIC_START; + + __ipipe_handle_irq(irq, regs); + } while (status); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + static void __init intcp_init_irq(void) { unsigned int i; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-13 14:31:28.000000000 +0200 @@ -16,6 +16,7 @@ #include <linux/module.h> #include <linux/interrupt.h> #include <linux/ptrace.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/irq.h> @@ -214,6 +215,42 @@ static void pxa_gpio_demux_handler(unsig } while (loop); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + unsigned int i, mask[4]; + int loop; + + do { + loop = 0; + + mask[0] = GEDR0 & ~3; + mask[1] = GEDR1; + mask[2] = GEDR2; +#if PXA_LAST_GPIO < 96 + i = 3; +#else /* PXA_LAST_GPIO >= 96 */ + mask[3] = GEDR3; + i = 4; +#endif /* PXA_LAST_GPIO >= 96 */ + for (; i; i--) { + loop ||= mask[i - 1]; + while (mask[i - 1]) { + irq = fls(mask[i - 1]) - 1; + mask[i - 1] &= ~(1 << irq); + irq = IRQ_GPIO((i - 1) * 32 + irq); + + __ipipe_handle_irq(irq, regs); + } + } + } while (loop); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + static void pxa_ack_muxed_gpio(unsigned int irq) { int gpio = irq - IRQ_GPIO(2) + 2; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c 2006-08-02 19:17:30.000000000 +0200 @@ -19,6 +19,7 @@ #include <linux/signal.h> #include <linux/errno.h> #include <linux/sched.h> +#include <linux/module.h> #include <asm/system.h> #include <asm/hardware.h> @@ -30,6 +31,23 @@ #include <asm/arch/pxa-regs.h> +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int pxa_timer_initialized; +static unsigned long pxa_jiffies; +#endif /* CONFIG_IPIPE */ + static inline unsigned long pxa_get_rtc_time(void) { return RCNR; @@ -54,6 +72,9 @@ static unsigned long pxa_gettimeoffset ( { long ticks_to_match, elapsed, usec; +#ifdef CONFIG_IPIPE + if (!__ipipe_mach_timerstolen) { +#endif /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; @@ -63,6 +84,10 @@ static unsigned long pxa_gettimeoffset ( /* don't get fooled by the workaround in pxa_timer_interrupt() */ if (elapsed <= 0) return 0; +#ifdef CONFIG_IPIPE + } else + elapsed = OSCR - pxa_jiffies * LATCH; +#endif /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; @@ -105,9 +130,27 @@ pxa_timer_interrupt(int irq, void *dev_i * affect things only when the timer IRQ has been delayed by nearly * exactly one tick period which should be a pretty rare event. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) { + timer_tick(regs); + ++pxa_jiffies; + } else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); +#ifdef CONFIG_IPIPE + ++pxa_jiffies; +#else /* !CONFIG_IPIPE */ OSSR = OSSR_M0; /* Clear match on timer 0 */ +#endif /* !CONFIG_IPIPE */ next_match = (OSMR0 += LATCH); } while( (signed long)(next_match - OSCR) <= 8 ); @@ -139,6 +182,10 @@ static void __init pxa_timer_init(void) setup_irq(IRQ_OST0, &pxa_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + pxa_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -216,3 +263,69 @@ struct sys_timer pxa_timer = { .dyn_tick = &pxa_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(pxa_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + if (delay > 8) { + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); + } else + ipipe_trigger_irq(IRQ_OST0); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/irq.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c 2006-08-03 19:05:03.000000000 +0200 @@ -14,6 +14,7 @@ #include <linux/ioport.h> #include <linux/ptrace.h> #include <linux/sysdev.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/irq.h> @@ -136,6 +137,37 @@ sa1100_high_gpio_handler(unsigned int ir } while (mask); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + unsigned int mask; + + mask = GEDR & 0xfffff800; + do { + /* + * clear down all currently active IRQ sources. + * We will be processing them all. + */ + GEDR = mask; + + irq = IRQ_GPIO11; + mask >>= 11; + do { + if (mask & 1) + __ipipe_handle_irq(irq, regs); + mask >>= 1; + irq++; + } while (mask); + + mask = GEDR & 0xfffff800; + } while (mask); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + /* * Like GPIO0 to 10, GPIO11-27 IRQs need to be handled specially. * In addition, the IRQs are all collected up into one bit in the diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c 2006-08-02 19:09:06.000000000 +0200 @@ -13,6 +13,7 @@ #include <linux/interrupt.h> #include <linux/timex.h> #include <linux/signal.h> +#include <linux/module.h> #include <asm/mach/time.h> #include <asm/hardware.h> @@ -20,6 +21,23 @@ #define RTC_DEF_DIVIDER (32768 - 1) #define RTC_DEF_TRIM 0 +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int sa1100_timer_initialized; +static unsigned long sa1100_jiffies; +#endif /* CONFIG_IPIPE */ + static unsigned long __init sa1100_get_rtc_time(void) { /* @@ -58,11 +76,18 @@ static unsigned long sa1100_gettimeoffse { unsigned long ticks_to_match, elapsed, usec; +#ifdef CONFIG_IPIPE + if (!__ipipe_mach_timerstolen) { +#endif /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; /* We need elapsed ticks since last match */ elapsed = LATCH - ticks_to_match; +#ifdef CONFIG_IPIPE + } else + elapsed = OSCR - sa1100_jiffies * LATCH; +#endif /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; @@ -97,9 +122,27 @@ sa1100_timer_interrupt(int irq, void *de * ensured, hence we can use do_gettimeofday() from interrupt * handlers. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) { + timer_tick(regs); + ++sa1100_jiffies; + } else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); +#ifdef CONFIG_IPIPE + ++sa1100_jiffies; +#else /* !CONFIG_IPIPE */ OSSR = OSSR_M0; /* Clear match on timer 0 */ +#endif /* !CONFIG_IPIPE */ next_match = (OSMR0 += LATCH); } while ((signed long)(next_match - OSCR) <= 0); @@ -131,6 +174,10 @@ static void __init sa1100_timer_init(voi setup_irq(IRQ_OST0, &sa1100_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + sa1100_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -209,3 +256,66 @@ struct sys_timer sa1100_timer = { .dyn_tick = &sa1100_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(sa1100_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-integrator/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-integrator/irqs.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/irqs.h 2006-08-03 18:17:55.000000000 +0200 @@ -80,3 +80,6 @@ #define NR_IRQS 47 +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_CP_CPPLDINT) +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-pxa/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-pxa/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-pxa/irqs.h 2006-07-15 20:06:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-pxa/irqs.h 2006-08-03 18:01:15.000000000 +0200 @@ -73,6 +73,10 @@ ((i) - IRQ_GPIO(2) + 2) #define IRQ_TO_GPIO(i) (((i) < IRQ_GPIO(2)) ? ((i) - IRQ_GPIO0) : IRQ_TO_GPIO_2_x(i)) +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_GPIO_2_x) +#endif /* CONFIG_IPIPE */ + #if defined(CONFIG_PXA25x) #define PXA_LAST_GPIO 80 #elif defined(CONFIG_PXA27x) diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-sa1100/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-sa1100/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-sa1100/irqs.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-sa1100/irqs.h 2006-08-03 18:14:34.000000000 +0200 @@ -145,6 +145,10 @@ #define IRQ_LOCOMO_SPI_OVRN (IRQ_BOARD_END + 20) #define IRQ_LOCOMO_SPI_TEND (IRQ_BOARD_END + 21) +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_GPIO11_27) +#endif /* CONFIG_IPIPE */ + /* * Figure out the MAX IRQ number. * diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/ipipe.h 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h 2006-08-03 17:52:02.000000000 +0200 @@ -117,6 +117,7 @@ extern void __ipipe_mach_acktimer(void); extern unsigned long long __ipipe_mach_get_tsc(void); extern void __ipipe_mach_set_dec(unsigned long); extern unsigned long __ipipe_mach_get_dec(void); +extern void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs); #define ipipe_read_tsc(t) do { t = __ipipe_mach_get_tsc(); } while (0) #define __ipipe_read_timebase() __ipipe_mach_get_tsc() diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/system.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/system.h 2006-08-04 14:52:36.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h 2006-07-17 18:40:59.000000000 +0200 @@ -322,7 +322,7 @@ void __ipipe_restore_root(unsigned long #define local_fiq_enable() __ipipe_unstall_root() #define local_fiq_disable() __ipipe_stall_root() #define local_save_flags(flags) ((flags) = __ipipe_test_root() << 7) -#define local_irq_restore(flags) __ipipe_restore_root(flags & PSR_I_BIT) +#define local_irq_restore(flags) __ipipe_restore_root(flags & (1 << 7)) #define irqs_disabled() __ipipe_test_root() @@ -413,17 +413,17 @@ static inline unsigned long __xchg(unsig #error SMP is not supported on this platform #endif case 1: - local_irq_save(flags); + local_irq_save_hw(flags); ret = *(volatile unsigned char *)ptr; *(volatile unsigned char *)ptr = x; - local_irq_restore(flags); + local_irq_restore_hw(flags); break; case 4: - local_irq_save(flags); + local_irq_save_hw(flags); ret = *(volatile unsigned long *)ptr; *(volatile unsigned long *)ptr = x; - local_irq_restore(flags); + local_irq_restore_hw(flags); break; #else case 1: ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-13 13:16 ` Gilles Chanteperdrix @ 2006-09-10 8:56 ` Detlef Vollmann 2006-09-10 12:19 ` Gilles Chanteperdrix 0 siblings, 1 reply; 36+ messages in thread From: Detlef Vollmann @ 2006-09-10 8:56 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai Hello, Gilles Chanteperdrix wrote: > Attached a new version of the patch (including the patch for the __xchg > operation, so that this patch can be applied incrementally after the > official I-pipe patch) which should be more greedy when demultiplexing > the GPIO interrupt. Thanks, great. Finally I came around to test it. There is one small typo in it: > diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c > --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-13 14:31:28.000000000 +0200 > @@ -214,6 +215,42 @@ static void pxa_gpio_demux_handler(unsig [...] > + for (; i; i--) { > + loop ||= mask[i - 1]; This should be: loop |= mask[i - 1]; With that change, first tests look fine. BTW, on the SA, what results do you get from xeno-test? Detlef -- Detlef Vollmann vollmann engineering gmbh Linux and C++ for Embedded Systems http://www.vollmann.ch/ Linux for PXA270 Colibri module: http://www.vollmann.ch/en/colibri/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-09-10 8:56 ` Detlef Vollmann @ 2006-09-10 12:19 ` Gilles Chanteperdrix 2006-09-10 16:31 ` Detlef Vollmann 0 siblings, 1 reply; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-09-10 12:19 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai [-- Attachment #1: message body and .signature --] [-- Type: text/plain, Size: 354 bytes --] Detlef Vollmann wrote: > With that change, first tests look fine. Thanks, attached the patch with this error corrected. > BTW, on the SA, what results do you get from xeno-test? latency in user-space easily get above 100 microseconds, if this is what you mean. But I guess this is what we should expect on ARM. -- Gilles Chanteperdrix. [-- Attachment #2: ipipe-sa1100-pxa.6.patch --] [-- Type: text/plain, Size: 17967 bytes --] diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/kernel/ipipe-root.c linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/kernel/ipipe-root.c 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/kernel/ipipe-root.c 2006-08-18 23:37:55.000000000 +0200 @@ -99,7 +99,11 @@ void __ipipe_enable_pipeline(void) ipipe_virtualize_irq(ipipe_root_domain, irq, (ipipe_irq_handler_t)&asm_do_IRQ, NULL, - (irq == __ipipe_mach_timerint) ? &__ipipe_ack_timerirq : &__ipipe_ack_irq, + ((irq == __ipipe_mach_timerint) + ? &__ipipe_ack_timerirq + : (irq_desc[irq].chip->ack + ? &__ipipe_ack_irq + : NULL)), IPIPE_HANDLE_MASK | IPIPE_PASS_MASK); /* @@ -302,7 +306,10 @@ asmlinkage int __ipipe_grab_irq(int irq, } } - __ipipe_handle_irq(irq, regs); + if (__ipipe_mach_irq_mux_p(irq)) + __ipipe_mach_demux_irq(irq, regs); + else + __ipipe_handle_irq(irq, regs); ipipe_load_cpuid(); diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-integrator/integrator_cp.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-integrator/integrator_cp.c 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-integrator/integrator_cp.c 2006-08-03 19:05:46.000000000 +0200 @@ -20,6 +20,7 @@ #include <linux/amba/bus.h> #include <linux/amba/kmi.h> #include <linux/amba/clcd.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/io.h> @@ -220,6 +221,31 @@ sic_handle_irq(unsigned int irq, struct } while (status); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + unsigned long status = sic_readl(INTCP_VA_SIC_BASE + IRQ_STATUS); + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + + if (status == 0) { + do_bad_IRQ(irq, desc_unused, regs); + return; + } + + do { + irq = ffs(status) - 1; + status &= ~(1 << irq); + + irq += IRQ_SIC_START; + + __ipipe_handle_irq(irq, regs); + } while (status); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + static void __init intcp_init_irq(void) { unsigned int i; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/irq.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-09-10 14:08:25.000000000 +0200 @@ -16,6 +16,7 @@ #include <linux/module.h> #include <linux/interrupt.h> #include <linux/ptrace.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/irq.h> @@ -214,6 +215,42 @@ static void pxa_gpio_demux_handler(unsig } while (loop); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + unsigned int i, mask[4]; + int loop; + + do { + loop = 0; + + mask[0] = GEDR0 & ~3; + mask[1] = GEDR1; + mask[2] = GEDR2; +#if PXA_LAST_GPIO < 96 + i = 3; +#else /* PXA_LAST_GPIO >= 96 */ + mask[3] = GEDR3; + i = 4; +#endif /* PXA_LAST_GPIO >= 96 */ + for (; i; i--) { + loop |= mask[i - 1]; + while (mask[i - 1]) { + irq = fls(mask[i - 1]) - 1; + mask[i - 1] &= ~(1 << irq); + irq = IRQ_GPIO((i - 1) * 32 + irq); + + __ipipe_handle_irq(irq, regs); + } + } + } while (loop); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + static void pxa_ack_muxed_gpio(unsigned int irq) { int gpio = irq - IRQ_GPIO(2) + 2; diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-pxa/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c 2006-08-02 19:17:30.000000000 +0200 @@ -19,6 +19,7 @@ #include <linux/signal.h> #include <linux/errno.h> #include <linux/sched.h> +#include <linux/module.h> #include <asm/system.h> #include <asm/hardware.h> @@ -30,6 +31,23 @@ #include <asm/arch/pxa-regs.h> +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int pxa_timer_initialized; +static unsigned long pxa_jiffies; +#endif /* CONFIG_IPIPE */ + static inline unsigned long pxa_get_rtc_time(void) { return RCNR; @@ -54,6 +72,9 @@ static unsigned long pxa_gettimeoffset ( { long ticks_to_match, elapsed, usec; +#ifdef CONFIG_IPIPE + if (!__ipipe_mach_timerstolen) { +#endif /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; @@ -63,6 +84,10 @@ static unsigned long pxa_gettimeoffset ( /* don't get fooled by the workaround in pxa_timer_interrupt() */ if (elapsed <= 0) return 0; +#ifdef CONFIG_IPIPE + } else + elapsed = OSCR - pxa_jiffies * LATCH; +#endif /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; @@ -105,9 +130,27 @@ pxa_timer_interrupt(int irq, void *dev_i * affect things only when the timer IRQ has been delayed by nearly * exactly one tick period which should be a pretty rare event. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) { + timer_tick(regs); + ++pxa_jiffies; + } else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); +#ifdef CONFIG_IPIPE + ++pxa_jiffies; +#else /* !CONFIG_IPIPE */ OSSR = OSSR_M0; /* Clear match on timer 0 */ +#endif /* !CONFIG_IPIPE */ next_match = (OSMR0 += LATCH); } while( (signed long)(next_match - OSCR) <= 8 ); @@ -139,6 +182,10 @@ static void __init pxa_timer_init(void) setup_irq(IRQ_OST0, &pxa_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + pxa_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -216,3 +263,69 @@ struct sys_timer pxa_timer = { .dyn_tick = &pxa_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(pxa_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + if (delay > 8) { + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); + } else + ipipe_trigger_irq(IRQ_OST0); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/irq.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/irq.c 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c 2006-08-03 19:05:03.000000000 +0200 @@ -14,6 +14,7 @@ #include <linux/ioport.h> #include <linux/ptrace.h> #include <linux/sysdev.h> +#include <linux/ipipe.h> #include <asm/hardware.h> #include <asm/irq.h> @@ -136,6 +137,37 @@ sa1100_high_gpio_handler(unsigned int ir } while (mask); } +#ifdef CONFIG_IPIPE +void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs) +{ + struct irqdesc *desc_unused = irq_desc + irq; + unsigned irq_unused = irq; + unsigned int mask; + + mask = GEDR & 0xfffff800; + do { + /* + * clear down all currently active IRQ sources. + * We will be processing them all. + */ + GEDR = mask; + + irq = IRQ_GPIO11; + mask >>= 11; + do { + if (mask & 1) + __ipipe_handle_irq(irq, regs); + mask >>= 1; + irq++; + } while (mask); + + mask = GEDR & 0xfffff800; + } while (mask); + + desc_unused->chip->unmask(irq_unused); +} +#endif /* CONFIG_IPIPE */ + /* * Like GPIO0 to 10, GPIO11-27 IRQs need to be handled specially. * In addition, the IRQs are all collected up into one bit in the diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/time.c linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c --- linux-2.6.16.5-tcl1-ipipe-ref/arch/arm/mach-sa1100/time.c 2006-05-07 15:36:35.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c 2006-08-02 19:09:06.000000000 +0200 @@ -13,6 +13,7 @@ #include <linux/interrupt.h> #include <linux/timex.h> #include <linux/signal.h> +#include <linux/module.h> #include <asm/mach/time.h> #include <asm/hardware.h> @@ -20,6 +21,23 @@ #define RTC_DEF_DIVIDER (32768 - 1) #define RTC_DEF_TRIM 0 +#ifdef CONFIG_IPIPE +#ifdef CONFIG_NO_IDLE_HZ +#error "dynamic tick timer not yet supported with IPIPE" +#endif /* CONFIG_NO_IDLE_HZ */ +int __ipipe_mach_timerint = IRQ_OST0; +EXPORT_SYMBOL(__ipipe_mach_timerint); + +int __ipipe_mach_timerstolen = 0; +EXPORT_SYMBOL(__ipipe_mach_timerstolen); + +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); + +static int sa1100_timer_initialized; +static unsigned long sa1100_jiffies; +#endif /* CONFIG_IPIPE */ + static unsigned long __init sa1100_get_rtc_time(void) { /* @@ -58,11 +76,18 @@ static unsigned long sa1100_gettimeoffse { unsigned long ticks_to_match, elapsed, usec; +#ifdef CONFIG_IPIPE + if (!__ipipe_mach_timerstolen) { +#endif /* Get ticks before next timer match */ ticks_to_match = OSMR0 - OSCR; /* We need elapsed ticks since last match */ elapsed = LATCH - ticks_to_match; +#ifdef CONFIG_IPIPE + } else + elapsed = OSCR - sa1100_jiffies * LATCH; +#endif /* Now convert them to usec */ usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; @@ -97,9 +122,27 @@ sa1100_timer_interrupt(int irq, void *de * ensured, hence we can use do_gettimeofday() from interrupt * handlers. */ +#ifdef CONFIG_IPIPE + /* + * - if Linux is running natively (no ipipe), ack and reprogram the timer + * - if Linux is running under ipipe, but it still has the control over + * the timer (no Xenomai for example), then reprogram the timer (ipipe + * has already acked it) + * - if some other domain has taken over the timer, then do nothing + * (ipipe has acked it, and the other domain has reprogramed it) + */ + if (__ipipe_mach_timerstolen) { + timer_tick(regs); + ++sa1100_jiffies; + } else +#endif /* CONFIG_IPIPE */ do { timer_tick(regs); +#ifdef CONFIG_IPIPE + ++sa1100_jiffies; +#else /* !CONFIG_IPIPE */ OSSR = OSSR_M0; /* Clear match on timer 0 */ +#endif /* !CONFIG_IPIPE */ next_match = (OSMR0 += LATCH); } while ((signed long)(next_match - OSCR) <= 0); @@ -131,6 +174,10 @@ static void __init sa1100_timer_init(voi setup_irq(IRQ_OST0, &sa1100_timer_irq); OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ OSCR = 0; /* initialize free-running timer */ + +#ifdef CONFIG_IPIPE + sa1100_timer_initialized = 1; +#endif /* CONFIG_IPIPE */ } #ifdef CONFIG_NO_IDLE_HZ @@ -209,3 +256,66 @@ struct sys_timer sa1100_timer = { .dyn_tick = &sa1100_dyn_tick, #endif }; + +#ifdef CONFIG_IPIPE +void __ipipe_mach_acktimer(void) +{ + OSSR = OSSR_M0; /* Clear match on timer 0 */ +} + +unsigned long long __ipipe_mach_get_tsc(void) +{ + if (likely(sa1100_timer_initialized)) { + static union { +#ifdef __BIG_ENDIAN + struct { + unsigned long high; + unsigned long low; + }; +#else /* __LITTLE_ENDIAN */ + struct { + unsigned long low; + unsigned long high; + }; +#endif /* __LITTLE_ENDIAN */ + unsigned long long full; + } tsc[NR_CPUS], *local_tsc; + unsigned long stamp, flags; + unsigned long long result; + + local_irq_save_hw(flags); + local_tsc = &tsc[ipipe_processor_id()]; + stamp = OSCR; + if (unlikely(stamp < local_tsc->low)) + /* 32 bit counter wrapped, increment high word. */ + local_tsc->high++; + local_tsc->low = stamp; + result = local_tsc->full; + local_irq_restore_hw(flags); + + return result; + } + + return 0; +} +EXPORT_SYMBOL(__ipipe_mach_get_tsc); + +/* + * Reprogram the timer + */ + +void __ipipe_mach_set_dec(unsigned long delay) +{ + unsigned long flags; + + local_irq_save_hw(flags); + OSMR0 = delay + OSCR; + local_irq_restore_hw(flags); +} +EXPORT_SYMBOL(__ipipe_mach_set_dec); + +unsigned long __ipipe_mach_get_dec(void) +{ + return OSMR0 - OSCR; +} +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-integrator/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-integrator/irqs.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-integrator/irqs.h 2006-08-03 18:17:55.000000000 +0200 @@ -80,3 +80,6 @@ #define NR_IRQS 47 +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_CP_CPPLDINT) +#endif /* CONFIG_IPIPE */ diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-pxa/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-pxa/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-pxa/irqs.h 2006-07-15 20:06:03.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-pxa/irqs.h 2006-08-03 18:01:15.000000000 +0200 @@ -73,6 +73,10 @@ ((i) - IRQ_GPIO(2) + 2) #define IRQ_TO_GPIO(i) (((i) < IRQ_GPIO(2)) ? ((i) - IRQ_GPIO0) : IRQ_TO_GPIO_2_x(i)) +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_GPIO_2_x) +#endif /* CONFIG_IPIPE */ + #if defined(CONFIG_PXA25x) #define PXA_LAST_GPIO 80 #elif defined(CONFIG_PXA27x) diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-sa1100/irqs.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-sa1100/irqs.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/arch-sa1100/irqs.h 2005-10-28 02:02:08.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/arch-sa1100/irqs.h 2006-08-03 18:14:34.000000000 +0200 @@ -145,6 +145,10 @@ #define IRQ_LOCOMO_SPI_OVRN (IRQ_BOARD_END + 20) #define IRQ_LOCOMO_SPI_TEND (IRQ_BOARD_END + 21) +#ifdef CONFIG_IPIPE +#define __ipipe_mach_irq_mux_p(irq) ((irq) == IRQ_GPIO11_27) +#endif /* CONFIG_IPIPE */ + /* * Figure out the MAX IRQ number. * diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/ipipe.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/ipipe.h 2006-08-03 18:08:48.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/ipipe.h 2006-08-03 17:52:02.000000000 +0200 @@ -117,6 +117,7 @@ extern void __ipipe_mach_acktimer(void); extern unsigned long long __ipipe_mach_get_tsc(void); extern void __ipipe_mach_set_dec(unsigned long); extern unsigned long __ipipe_mach_get_dec(void); +extern void __ipipe_mach_demux_irq(unsigned irq, struct pt_regs *regs); #define ipipe_read_tsc(t) do { t = __ipipe_mach_get_tsc(); } while (0) #define __ipipe_read_timebase() __ipipe_mach_get_tsc() diff -Naurdp -x '*~' -x '*.orig' -x '*.rej' linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/system.h linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h --- linux-2.6.16.5-tcl1-ipipe-ref/include/asm-arm/system.h 2006-08-04 14:52:36.000000000 +0200 +++ linux-2.6.16.5-tcl1-ipipe/include/asm-arm/system.h 2006-07-17 18:40:59.000000000 +0200 @@ -322,7 +322,7 @@ void __ipipe_restore_root(unsigned long #define local_fiq_enable() __ipipe_unstall_root() #define local_fiq_disable() __ipipe_stall_root() #define local_save_flags(flags) ((flags) = __ipipe_test_root() << 7) -#define local_irq_restore(flags) __ipipe_restore_root(flags & PSR_I_BIT) +#define local_irq_restore(flags) __ipipe_restore_root(flags & (1 << 7)) #define irqs_disabled() __ipipe_test_root() @@ -413,17 +413,17 @@ static inline unsigned long __xchg(unsig #error SMP is not supported on this platform #endif case 1: - local_irq_save(flags); + local_irq_save_hw(flags); ret = *(volatile unsigned char *)ptr; *(volatile unsigned char *)ptr = x; - local_irq_restore(flags); + local_irq_restore_hw(flags); break; case 4: - local_irq_save(flags); + local_irq_save_hw(flags); ret = *(volatile unsigned long *)ptr; *(volatile unsigned long *)ptr = x; - local_irq_restore(flags); + local_irq_restore_hw(flags); break; #else case 1: ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-09-10 12:19 ` Gilles Chanteperdrix @ 2006-09-10 16:31 ` Detlef Vollmann 2006-09-10 16:37 ` Gilles Chanteperdrix 0 siblings, 1 reply; 36+ messages in thread From: Detlef Vollmann @ 2006-09-10 16:31 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai Gilles Chanteperdrix wrote: > Detlef Vollmann wrote: > > With that change, first tests look fine. > > Thanks, attached the patch with this error corrected. Thanks. > > BTW, on the SA, what results do you get from xeno-test? > > latency in user-space easily get above 100 microseconds, if this is what > you mean. But I guess this is what we should expect on ARM. Probably yes. The histogram just looks a bit strange, nearly all in the last column... Another point: did you run the switchbench test? With '-n 100000' I get a sure softlock. Detlef -- Detlef Vollmann vollmann engineering gmbh Linux and C++ for Embedded Systems http://www.vollmann.ch/ Linux for PXA270 Colibri module: http://www.vollmann.ch/en/colibri/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-09-10 16:31 ` Detlef Vollmann @ 2006-09-10 16:37 ` Gilles Chanteperdrix 2006-09-10 19:20 ` Detlef Vollmann 0 siblings, 1 reply; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-09-10 16:37 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai Detlef Vollmann wrote: > Gilles Chanteperdrix wrote: > > Detlef Vollmann wrote: > > > With that change, first tests look fine. > > > > Thanks, attached the patch with this error corrected. > Thanks. > > > > BTW, on the SA, what results do you get from xeno-test? > > > > latency in user-space easily get above 100 microseconds, if this is what > > you mean. But I guess this is what we should expect on ARM. > Probably yes. The histogram just looks a bit strange, nearly > all in the last column... > > Another point: did you run the switchbench test? > > With '-n 100000' I get a sure softlock. Probably the default sampling period of 100 us is too hard for ARM, Xenomai never gets out of the timer interrupt handler, you should try and pass -p 1000. -- Gilles Chanteperdrix. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-09-10 16:37 ` Gilles Chanteperdrix @ 2006-09-10 19:20 ` Detlef Vollmann 2006-09-10 19:42 ` Gilles Chanteperdrix 0 siblings, 1 reply; 36+ messages in thread From: Detlef Vollmann @ 2006-09-10 19:20 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai Gilles Chanteperdrix wrote: > Detlef Vollmann wrote: > > Another point: did you run the switchbench test? > > > > With '-n 100000' I get a sure softlock. > > Probably the default sampling period of 100 us is too hard for ARM, Maybe, but I'm not sure (my PXA runs at 520MHz). > Xenomai never gets out of the timer interrupt handler, I don't think that's the problem. With '-n 1000', everything is fine. With '-n 10000', on a non-loaded machine, everything is still fine, but on a loaded machine, I get the softlock. With '-n 30000' even on an unloaded machine I get a softlock most of the time. I think I'll go and try to understand what switchbench actually does and what happens on that softlock. Then I'll probably come back with more questions. Detlef -- Detlef Vollmann vollmann engineering gmbh Linux and C++ for Embedded Systems http://www.vollmann.ch/ Linux for PXA270 Colibri module: http://www.vollmann.ch/en/colibri/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-09-10 19:20 ` Detlef Vollmann @ 2006-09-10 19:42 ` Gilles Chanteperdrix 0 siblings, 0 replies; 36+ messages in thread From: Gilles Chanteperdrix @ 2006-09-10 19:42 UTC (permalink / raw) To: Detlef Vollmann; +Cc: xenomai Detlef Vollmann wrote: > Gilles Chanteperdrix wrote: > > Detlef Vollmann wrote: > > > Another point: did you run the switchbench test? > > > > > > With '-n 100000' I get a sure softlock. > > > > Probably the default sampling period of 100 us is too hard for ARM, > Maybe, but I'm not sure (my PXA runs at 520MHz). > > > Xenomai never gets out of the timer interrupt handler, > I don't think that's the problem. > With '-n 1000', everything is fine. > With '-n 10000', on a non-loaded machine, everything is still fine, > but on a loaded machine, I get the softlock. > With '-n 30000' even on an unloaded machine I get a softlock > most of the time. > > I think I'll go and try to understand what switchbench actually > does and what happens on that softlock. Then I'll probably come > back with more questions. You should try -p 1000. Really. -- Gilles Chanteperdrix. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-08-02 18:06 ` Gilles Chanteperdrix 2006-08-03 9:12 ` Bart Jonkers @ 2006-08-03 11:23 ` Bart Jonkers 1 sibling, 0 replies; 36+ messages in thread From: Bart Jonkers @ 2006-08-03 11:23 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai On Wed, 2006-08-02 at 20:06 +0200, Gilles Chanteperdrix wrote: > Bart Jonkers wrote: > > On Wed, 2006-08-02 at 15:09 +0200, Gilles Chanteperdrix wrote: > > > Detlef Vollmann wrote: > > > > > I also added an interrupt handler on a button which is connected to GPIO > > > > > pin. The interrupt handler should print something when it is executed. > > > > > When I push the button a couple of times nothing happens. So GPIO > > > > > interrupts doesn't seems to work. > > > > Probably GPIO0 and GPIO1 work. > > > > I suspect it's the cascading interrupt that doesn't work. > > > > When I played around with an earlier version of ipipe, I found that > > > > the IRQ_GPIO_2_80 (IRQ 10 on PXA270) is masked, but never unmasked > > > > again. > > > > > > > > The problem is that for the cascading interrupt you need a special > > > > IPIPE handler, and I don't think there is currently one for PXA. > > > > > > > > How is that done on other machines/architectures? > > > > > > When looking at the ipipe_enable_pipeline function, we see that > > > interrupts management routines are all intercepted by the I-pipe, so the > > > cascaded interrupts management routines should be automatically > > > intercepted. > > > > > > In order to have a better understanding of the issue, it would be > > > interesting if you could trace the functions that are called on the path > > > from the interrupt to the execution of the final handler. > > > > I have tracked and solved the issue. I have looked to the i.MX21 port > > and they added some code to the GPIO interrupt handler of the i.MX21. > > When IPIPE is active they unmask the interrupt for the GPIO pins at the > > end of the handler. I did the same for PXA and my GPIO interrupt problem > > is solved. I think that the same is needed for SA-1100. > > Here is a new version of the ipipe-sa1100-pxa patch that unmaks > interrupts at the end of the demux handlers, and that attempt to fix the > gettimeoffset issue. I have run 20 minutes (time for OSCR to wrap) latency > with a test program verifying that time returned by gettimeofday is > never going backward. > > The patch and the test program are attached, it would be nice if you > could test them. > To let the GPIO interrupts also work in the real-time domain we need to set up a chain handler for these interrupts. The problem is that I know very less of the internals of xenomai to implement this. So could someone help me with this? I've already take a look to the i.mx21 platform but doesn't see anything that could help us. Maybe GPIO interrupts also doesn't work on the i.mx21 platform. Could the people that work on this platform confirm that? Thanks, Bart > plain text document attachment (ipipe-sa1100-pxa.2.patch) > --- linux-2.6.16.5-tcl1/arch/arm/mach-pxa/irq.c 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/irq.c 2006-08-02 17:57:30.000000000 +0200 > @@ -143,6 +143,10 @@ static struct irqchip pxa_low_gpio_chip > static void pxa_gpio_demux_handler(unsigned int irq, struct irqdesc *desc, > struct pt_regs *regs) > { > +#ifdef CONFIG_IPIPE > + struct irqdesc *desc_unused = desc; > + unsigned irq_unused = irq; > +#endif /* CONFIG_IPIPE */ > unsigned int mask; > int loop; > > @@ -212,6 +216,10 @@ static void pxa_gpio_demux_handler(unsig > } > #endif > } while (loop); > + > +#ifdef CONFIG_IPIPE > + desc_unused->chip->unmask(irq_unused); > +#endif /* CONFIG_IPIPE */ > } > > static void pxa_ack_muxed_gpio(unsigned int irq) > --- linux-2.6.16.5-tcl1/arch/arm/mach-pxa/time.c 2006-05-07 15:36:35.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-pxa/time.c 2006-08-02 19:17:30.000000000 +0200 > @@ -19,6 +19,7 @@ > #include <linux/signal.h> > #include <linux/errno.h> > #include <linux/sched.h> > +#include <linux/module.h> > > #include <asm/system.h> > #include <asm/hardware.h> > @@ -30,6 +31,23 @@ > #include <asm/arch/pxa-regs.h> > > > +#ifdef CONFIG_IPIPE > +#ifdef CONFIG_NO_IDLE_HZ > +#error "dynamic tick timer not yet supported with IPIPE" > +#endif /* CONFIG_NO_IDLE_HZ */ > +int __ipipe_mach_timerint = IRQ_OST0; > +EXPORT_SYMBOL(__ipipe_mach_timerint); > + > +int __ipipe_mach_timerstolen = 0; > +EXPORT_SYMBOL(__ipipe_mach_timerstolen); > + > +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; > +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); > + > +static int pxa_timer_initialized; > +static unsigned long pxa_jiffies; > +#endif /* CONFIG_IPIPE */ > + > static inline unsigned long pxa_get_rtc_time(void) > { > return RCNR; > @@ -54,6 +72,9 @@ static unsigned long pxa_gettimeoffset ( > { > long ticks_to_match, elapsed, usec; > > +#ifdef CONFIG_IPIPE > + if (!__ipipe_mach_timerstolen) { > +#endif > /* Get ticks before next timer match */ > ticks_to_match = OSMR0 - OSCR; > > @@ -63,6 +84,10 @@ static unsigned long pxa_gettimeoffset ( > /* don't get fooled by the workaround in pxa_timer_interrupt() */ > if (elapsed <= 0) > return 0; > +#ifdef CONFIG_IPIPE > + } else > + elapsed = OSCR - pxa_jiffies * LATCH; > +#endif > > /* Now convert them to usec */ > usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; > @@ -105,9 +130,27 @@ pxa_timer_interrupt(int irq, void *dev_i > * affect things only when the timer IRQ has been delayed by nearly > * exactly one tick period which should be a pretty rare event. > */ > +#ifdef CONFIG_IPIPE > + /* > + * - if Linux is running natively (no ipipe), ack and reprogram the timer > + * - if Linux is running under ipipe, but it still has the control over > + * the timer (no Xenomai for example), then reprogram the timer (ipipe > + * has already acked it) > + * - if some other domain has taken over the timer, then do nothing > + * (ipipe has acked it, and the other domain has reprogramed it) > + */ > + if (__ipipe_mach_timerstolen) { > + timer_tick(regs); > + ++pxa_jiffies; > + } else > +#endif /* CONFIG_IPIPE */ > do { > timer_tick(regs); > +#ifdef CONFIG_IPIPE > + ++pxa_jiffies; > +#else /* !CONFIG_IPIPE */ > OSSR = OSSR_M0; /* Clear match on timer 0 */ > +#endif /* !CONFIG_IPIPE */ > next_match = (OSMR0 += LATCH); > } while( (signed long)(next_match - OSCR) <= 8 ); > > @@ -139,6 +182,10 @@ static void __init pxa_timer_init(void) > setup_irq(IRQ_OST0, &pxa_timer_irq); > OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ > OSCR = 0; /* initialize free-running timer */ > + > +#ifdef CONFIG_IPIPE > + pxa_timer_initialized = 1; > +#endif /* CONFIG_IPIPE */ > } > > #ifdef CONFIG_NO_IDLE_HZ > @@ -216,3 +263,69 @@ struct sys_timer pxa_timer = { > .dyn_tick = &pxa_dyn_tick, > #endif > }; > + > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_acktimer(void) > +{ > + OSSR = OSSR_M0; /* Clear match on timer 0 */ > +} > + > +unsigned long long __ipipe_mach_get_tsc(void) > +{ > + if (likely(pxa_timer_initialized)) { > + static union { > +#ifdef __BIG_ENDIAN > + struct { > + unsigned long high; > + unsigned long low; > + }; > +#else /* __LITTLE_ENDIAN */ > + struct { > + unsigned long low; > + unsigned long high; > + }; > +#endif /* __LITTLE_ENDIAN */ > + unsigned long long full; > + } tsc[NR_CPUS], *local_tsc; > + unsigned long stamp, flags; > + unsigned long long result; > + > + local_irq_save_hw(flags); > + local_tsc = &tsc[ipipe_processor_id()]; > + stamp = OSCR; > + if (unlikely(stamp < local_tsc->low)) > + /* 32 bit counter wrapped, increment high word. */ > + local_tsc->high++; > + local_tsc->low = stamp; > + result = local_tsc->full; > + local_irq_restore_hw(flags); > + > + return result; > + } > + > + return 0; > +} > +EXPORT_SYMBOL(__ipipe_mach_get_tsc); > + > +/* > + * Reprogram the timer > + */ > + > +void __ipipe_mach_set_dec(unsigned long delay) > +{ > + if (delay > 8) { > + unsigned long flags; > + > + local_irq_save_hw(flags); > + OSMR0 = delay + OSCR; > + local_irq_restore_hw(flags); > + } else > + ipipe_trigger_irq(IRQ_OST0); > +} > +EXPORT_SYMBOL(__ipipe_mach_set_dec); > + > +unsigned long __ipipe_mach_get_dec(void) > +{ > + return OSMR0 - OSCR; > +} > +#endif /* CONFIG_IPIPE */ > --- linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/irq.c 2005-10-28 02:02:08.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/irq.c 2006-08-02 17:55:48.000000000 +0200 > @@ -111,6 +111,10 @@ static void > sa1100_high_gpio_handler(unsigned int irq, struct irqdesc *desc, > struct pt_regs *regs) > { > +#ifdef CONFIG_IPIPE > + struct irqdesc *desc_unused = desc; > + unsigned irq_unused = irq; > +#endif /* CONFIG_IPIPE */ > unsigned int mask; > > mask = GEDR & 0xfffff800; > @@ -134,6 +138,10 @@ sa1100_high_gpio_handler(unsigned int ir > > mask = GEDR & 0xfffff800; > } while (mask); > + > +#ifdef CONFIG_IPIPE > + desc_unused->chip->unmask(irq_unused); > +#endif /* CONFIG_IPIPE */ > } > > /* > --- linux-2.6.16.5-tcl1/arch/arm/mach-sa1100/time.c 2006-05-07 15:36:35.000000000 +0200 > +++ linux-2.6.16.5-tcl1-ipipe/arch/arm/mach-sa1100/time.c 2006-08-02 19:09:06.000000000 +0200 > @@ -13,6 +13,7 @@ > #include <linux/interrupt.h> > #include <linux/timex.h> > #include <linux/signal.h> > +#include <linux/module.h> > > #include <asm/mach/time.h> > #include <asm/hardware.h> > @@ -20,6 +21,23 @@ > #define RTC_DEF_DIVIDER (32768 - 1) > #define RTC_DEF_TRIM 0 > > +#ifdef CONFIG_IPIPE > +#ifdef CONFIG_NO_IDLE_HZ > +#error "dynamic tick timer not yet supported with IPIPE" > +#endif /* CONFIG_NO_IDLE_HZ */ > +int __ipipe_mach_timerint = IRQ_OST0; > +EXPORT_SYMBOL(__ipipe_mach_timerint); > + > +int __ipipe_mach_timerstolen = 0; > +EXPORT_SYMBOL(__ipipe_mach_timerstolen); > + > +unsigned int __ipipe_mach_ticks_per_jiffy = LATCH; > +EXPORT_SYMBOL(__ipipe_mach_ticks_per_jiffy); > + > +static int sa1100_timer_initialized; > +static unsigned long sa1100_jiffies; > +#endif /* CONFIG_IPIPE */ > + > static unsigned long __init sa1100_get_rtc_time(void) > { > /* > @@ -58,11 +76,18 @@ static unsigned long sa1100_gettimeoffse > { > unsigned long ticks_to_match, elapsed, usec; > > +#ifdef CONFIG_IPIPE > + if (!__ipipe_mach_timerstolen) { > +#endif > /* Get ticks before next timer match */ > ticks_to_match = OSMR0 - OSCR; > > /* We need elapsed ticks since last match */ > elapsed = LATCH - ticks_to_match; > +#ifdef CONFIG_IPIPE > + } else > + elapsed = OSCR - sa1100_jiffies * LATCH; > +#endif > > /* Now convert them to usec */ > usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH; > @@ -97,9 +122,27 @@ sa1100_timer_interrupt(int irq, void *de > * ensured, hence we can use do_gettimeofday() from interrupt > * handlers. > */ > +#ifdef CONFIG_IPIPE > + /* > + * - if Linux is running natively (no ipipe), ack and reprogram the timer > + * - if Linux is running under ipipe, but it still has the control over > + * the timer (no Xenomai for example), then reprogram the timer (ipipe > + * has already acked it) > + * - if some other domain has taken over the timer, then do nothing > + * (ipipe has acked it, and the other domain has reprogramed it) > + */ > + if (__ipipe_mach_timerstolen) { > + timer_tick(regs); > + ++sa1100_jiffies; > + } else > +#endif /* CONFIG_IPIPE */ > do { > timer_tick(regs); > +#ifdef CONFIG_IPIPE > + ++sa1100_jiffies; > +#else /* !CONFIG_IPIPE */ > OSSR = OSSR_M0; /* Clear match on timer 0 */ > +#endif /* !CONFIG_IPIPE */ > next_match = (OSMR0 += LATCH); > } while ((signed long)(next_match - OSCR) <= 0); > > @@ -131,6 +174,10 @@ static void __init sa1100_timer_init(voi > setup_irq(IRQ_OST0, &sa1100_timer_irq); > OIER = OIER_E0; /* enable match on timer 0 to cause interrupts */ > OSCR = 0; /* initialize free-running timer */ > + > +#ifdef CONFIG_IPIPE > + sa1100_timer_initialized = 1; > +#endif /* CONFIG_IPIPE */ > } > > #ifdef CONFIG_NO_IDLE_HZ > @@ -209,3 +256,66 @@ struct sys_timer sa1100_timer = { > .dyn_tick = &sa1100_dyn_tick, > #endif > }; > + > +#ifdef CONFIG_IPIPE > +void __ipipe_mach_acktimer(void) > +{ > + OSSR = OSSR_M0; /* Clear match on timer 0 */ > +} > + > +unsigned long long __ipipe_mach_get_tsc(void) > +{ > + if (likely(sa1100_timer_initialized)) { > + static union { > +#ifdef __BIG_ENDIAN > + struct { > + unsigned long high; > + unsigned long low; > + }; > +#else /* __LITTLE_ENDIAN */ > + struct { > + unsigned long low; > + unsigned long high; > + }; > +#endif /* __LITTLE_ENDIAN */ > + unsigned long long full; > + } tsc[NR_CPUS], *local_tsc; > + unsigned long stamp, flags; > + unsigned long long result; > + > + local_irq_save_hw(flags); > + local_tsc = &tsc[ipipe_processor_id()]; > + stamp = OSCR; > + if (unlikely(stamp < local_tsc->low)) > + /* 32 bit counter wrapped, increment high word. */ > + local_tsc->high++; > + local_tsc->low = stamp; > + result = local_tsc->full; > + local_irq_restore_hw(flags); > + > + return result; > + } > + > + return 0; > +} > +EXPORT_SYMBOL(__ipipe_mach_get_tsc); > + > +/* > + * Reprogram the timer > + */ > + > +void __ipipe_mach_set_dec(unsigned long delay) > +{ > + unsigned long flags; > + > + local_irq_save_hw(flags); > + OSMR0 = delay + OSCR; > + local_irq_restore_hw(flags); > +} > +EXPORT_SYMBOL(__ipipe_mach_set_dec); > + > +unsigned long __ipipe_mach_get_dec(void) > +{ > + return OSMR0 - OSCR; > +} > +#endif /* CONFIG_IPIPE */ > plain text document attachment (test_gettimeofday.c) > #include <stdio.h> > #include <time.h> > #include <sys/time.h> > > int main(void) > { > struct timeval tv, old_tv; > > gettimeofday(&old_tv, NULL); > for (;;) { > gettimeofday(&tv, NULL); > if (tv.tv_sec < old_tv.tv_sec || > (tv.tv_sec == old_tv.tv_sec && tv.tv_usec < old_tv.tv_usec)) > printf("Time is going backward !\n"); > old_tv = tv; > } > } ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Xenomai-core] Xenomai on PXA 2006-07-17 16:17 ` Gilles Chanteperdrix 2006-07-17 16:29 ` Philippe Gerum @ 2006-07-17 22:33 ` Danilo Levantesi 1 sibling, 0 replies; 36+ messages in thread From: Danilo Levantesi @ 2006-07-17 22:33 UTC (permalink / raw) To: xenomai [-- Attachment #1: Type: text/plain, Size: 1142 bytes --] > Starting from Detlef patch, I made some tests on an SA1100 based ARM, > attached is a patch that works for me, which I also adapted to PXA > without testing it. It would be nice if someone could test it on PXA. > > In order to do these tests, I had to adapt the ARM ipipe 2.6.15 patch to > linux 2.6.16, so the 2.6.16 ipipe patch is attached too for further > testing. Note that I had to make a small modification in > include/asm-arm/system.h, because the SA1100 version of __xchg uses > local_irq_restore, which use PSR_I_BIT, which is defined elsewhere, so I > replaced PSR_I_BIT with its value. I now wonder if it is the proper fix, > or if __xchg should be fixed to use local_irq_save_hw and > local_irq_restore_hw instead, as is the case for the atomic operations > defined in atomic.h and bitops.h. I'm sorry, but probably i can't test it, because i need an handelds.org patched kernel to load linux into my ipaq (i need the mmc driver), and the latest stable version is 2.4.19-rmk6-pxa1-hh41... No official support for 2.6 :'( BTW, if i can find a working 2.6.16 kernel i'll give it a try! Danilo Levantesi [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* [Xenomai-core] Xenomai on PXA @ 2007-10-17 16:29 Patrick 0 siblings, 0 replies; 36+ messages in thread From: Patrick @ 2007-10-17 16:29 UTC (permalink / raw) To: 'xenomai-core' [-- Attachment #1: Type: text/plain, Size: 432 bytes --] Hi, Has anyone successfully installed and used xenomai on PXA machine? If so, can you please tell me which version of xenomai & adios was used, and which version linux kernel you used ? I know that version 2.3.1 of xenomai with kernel 2.6.15 works, and that version 2.3.4 with kernel 2.6.20 does not work, but between two? I'm trying to find out at which versions things break. Thank you and best regards, Patrick [-- Attachment #2: Type: text/html, Size: 2789 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
end of thread, other threads:[~2007-10-17 16:29 UTC | newest] Thread overview: 36+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2006-07-07 15:02 [Xenomai-core] Xenomai on PXA Danilo Levantesi 2006-07-07 20:32 ` Gilles Chanteperdrix 2006-07-11 6:20 ` Detlef Vollmann 2006-07-11 15:01 ` Stelian Pop 2006-07-12 13:13 ` Bart Jonkers 2006-07-11 15:02 ` Stelian Pop 2006-07-17 16:17 ` Gilles Chanteperdrix 2006-07-17 16:29 ` Philippe Gerum 2006-07-17 16:56 ` Gilles Chanteperdrix 2006-07-31 8:34 ` Bart Jonkers 2006-07-31 9:20 ` Detlef Vollmann 2006-07-31 10:33 ` Bart Jonkers 2006-07-31 11:08 ` Detlef Vollmann 2006-08-02 13:09 ` Gilles Chanteperdrix 2006-08-02 13:33 ` Bart Jonkers 2006-08-02 13:56 ` Gilles Chanteperdrix 2006-08-02 18:06 ` Gilles Chanteperdrix 2006-08-03 9:12 ` Bart Jonkers 2006-08-03 13:18 ` Gilles Chanteperdrix 2006-08-03 14:56 ` Philippe Gerum 2006-08-03 17:14 ` Gilles Chanteperdrix 2006-08-04 6:10 ` Bart Jonkers 2006-08-04 6:10 ` Detlef Vollmann 2006-08-04 9:04 ` Philippe Gerum 2006-08-04 11:37 ` Gilles Chanteperdrix 2006-08-04 14:52 ` Bart Jonkers 2006-08-13 13:16 ` Gilles Chanteperdrix 2006-09-10 8:56 ` Detlef Vollmann 2006-09-10 12:19 ` Gilles Chanteperdrix 2006-09-10 16:31 ` Detlef Vollmann 2006-09-10 16:37 ` Gilles Chanteperdrix 2006-09-10 19:20 ` Detlef Vollmann 2006-09-10 19:42 ` Gilles Chanteperdrix 2006-08-03 11:23 ` Bart Jonkers 2006-07-17 22:33 ` Danilo Levantesi -- strict thread matches above, loose matches on Subject: below -- 2007-10-17 16:29 Patrick
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.