Missing some interrupts

Missing some interrupts

[wael showair]

Hi All,
i have a freescale board, that contains MPC8555 processor & DSP-core
there is a GPIO connecting the DSP-core into an input pin of the OpenPIC of
the MPC8555 processor.

i test one interrupt from the DSP-core to the MPC8555 processor where i
configure this interrupt line to be edge-triggered (falling edge) & i
receive it successfully 
but 
when i generate this interrupt 10 successive times using for loop
i just receive 2 interrupts?

why can't i receive the 8 other interrupts?
i print the value of every irq number inside do_IRQ & i found that i receive
the DSP-interrupt just only twice.

do u have any suggestions to solve this problem?
i want to make sure that i can receive 10 interrupts
-- 
View this message in context: http://www.nabble.com/Missing-some-interrupts-tp23901807p23901807.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Missing some interrupts

[Jon Smirl]

On Sat, Jun 6, 2009 at 9:17 AM, wael showair<showair2003@yahoo.com> wrote:
>
> Hi All,
> i have a freescale board, that contains MPC8555 processor & DSP-core
> there is a GPIO connecting the DSP-core into an input pin of the OpenPIC of
> the MPC8555 processor.
>
> i test one interrupt from the DSP-core to the MPC8555 processor where i
> configure this interrupt line to be edge-triggered (falling edge) & i
> receive it successfully
> but
> when i generate this interrupt 10 successive times using for loop
> i just receive 2 interrupts?
>
> why can't i receive the 8 other interrupts?
> i print the value of every irq number inside do_IRQ & i found that i receive
> the DSP-interrupt just only twice.
>
> do u have any suggestions to solve this problem?
> i want to make sure that i can receive 10 interrupts

Don't do a printk() with interrupts disabled. A printk() takes 1.5ms
on a mpc5200.


> --
> View this message in context: http://www.nabble.com/Missing-some-interrupts-tp23901807p23901807.html
> Sent from the linuxppc-dev mailing list archive at Nabble.com.
>
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
>



-- 
Jon Smirl
jonsmirl@gmail.com

Re: Missing some interrupts

[Grant Likely]

On Sat, Jun 6, 2009 at 7:34 AM, Jon Smirl<jonsmirl@gmail.com> wrote:
> On Sat, Jun 6, 2009 at 9:17 AM, wael showair<showair2003@yahoo.com> wrote:
>>
>> Hi All,
>> i have a freescale board, that contains MPC8555 processor & DSP-core
>> there is a GPIO connecting the DSP-core into an input pin of the OpenPIC of
>> the MPC8555 processor.
>>
>> i test one interrupt from the DSP-core to the MPC8555 processor where i
>> configure this interrupt line to be edge-triggered (falling edge) & i
>> receive it successfully
>> but
>> when i generate this interrupt 10 successive times using for loop
>> i just receive 2 interrupts?
>>
>> why can't i receive the 8 other interrupts?
>> i print the value of every irq number inside do_IRQ & i found that i receive
>> the DSP-interrupt just only twice.
>>
>> do u have any suggestions to solve this problem?
>> i want to make sure that i can receive 10 interrupts
>
> Don't do a printk() with interrupts disabled. A printk() takes 1.5ms
> on a mpc5200.

Look at /proc/interrupts instead to see how many IRQs are received.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Missing some interrupts

[Benjamin Herrenschmidt]

On Sat, 2009-06-06 at 06:17 -0700, wael showair wrote:
> Hi All,
> i have a freescale board, that contains MPC8555 processor & DSP-core
> there is a GPIO connecting the DSP-core into an input pin of the OpenPIC of
> the MPC8555 processor.
> 
> i test one interrupt from the DSP-core to the MPC8555 processor where i
> configure this interrupt line to be edge-triggered (falling edge) & i
> receive it successfully 
> but 
> when i generate this interrupt 10 successive times using for loop
> i just receive 2 interrupts?
> 
> why can't i receive the 8 other interrupts?
> i print the value of every irq number inside do_IRQ & i found that i receive
> the DSP-interrupt just only twice.

That sounds normal... It all depends what you are doing in the interrupt
handler. If you are doing something for too long, you will "miss" some
interrupts, but that isn't necessarily a problem.

You cannot really rely on getting the exact same number of edge
interrupts that were emitted. At least not unless you have a hard RT
system and can guarantee that you'll always dequeue them fast enough.

Basically, what happens is that in a PIC like the MPIC, if one edge
interrupt is latched, and another one arrives before that first one has
been acked, then the second one is "subsumed", ie, there's only one
input latch.

That should however not be a problem if your driver is written properly.
The idea is that when you get the interrupt, you need to check your
device for -all- the work to do, not only one "item". For example, if
the device fills a ring buffer, you need to check for more than one
entry in there.

The only guarantee you have is that the interrupt will have been acked
before your handler is called. So if another interrupt happens while
your handler is running, you -will- be called again. So you don't need
to worry too much about racing with new incoming messages inside the
interrupt handler itself. But you need to be prepared to pick up more
than one item of work... whatever that is.

Cheers,
Ben.

Re: Missing some interrupts

[wael showair]

Benjamin Herrenschmidt wrote:
> 
> On Sat, 2009-06-06 at 06:17 -0700, wael showair wrote:
>> Hi All,
>> i have a freescale board, that contains MPC8555 processor & DSP-core
>> there is a GPIO connecting the DSP-core into an input pin of the OpenPIC
>> of
>> the MPC8555 processor.
>> 
>> i test one interrupt from the DSP-core to the MPC8555 processor where i
>> configure this interrupt line to be edge-triggered (falling edge) & i
>> receive it successfully 
>> but 
>> when i generate this interrupt 10 successive times using for loop
>> i just receive 2 interrupts?
>> 
>> why can't i receive the 8 other interrupts?
>> i print the value of every irq number inside do_IRQ & i found that i
>> receive
>> the DSP-interrupt just only twice.
> 
> That sounds normal... It all depends what you are doing in the interrupt
> handler. If you are doing something for too long, you will "miss" some
> interrupts, but that isn't necessarily a problem.
> 
> You cannot really rely on getting the exact same number of edge
> interrupts that were emitted. At least not unless you have a hard RT
> system and can guarantee that you'll always dequeue them fast enough.
> 
> Basically, what happens is that in a PIC like the MPIC, if one edge
> interrupt is latched, and another one arrives before that first one has
> been acked, then the second one is "subsumed", ie, there's only one
> input latch.
> 
> That should however not be a problem if your driver is written properly.
> The idea is that when you get the interrupt, you need to check your
> device for -all- the work to do, not only one "item". For example, if
> the device fills a ring buffer, you need to check for more than one
> entry in there.
> 
> The only guarantee you have is that the interrupt will have been acked
> before your handler is called. So if another interrupt happens while
> your handler is running, you -will- be called again. 
> 
> So How can i achieve this? how can i ack the interrupt b 4 calling the
> handler?
> 
> So you don't need
> to worry too much about racing with new incoming messages inside the
> interrupt handler itself. But you need to be prepared to pick up more
> than one item of work... whatever that is.
> 
> Cheers,
> Ben.
> 
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
> 
> 

-- 
View this message in context: http://www.nabble.com/Missing-some-interrupts-tp23901807p23906763.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Missing some interrupts

[wael showair]

> > You cannot really rely on getting the exact same number of edge
> > interrupts that were emitted. At least not unless you have a hard RT
> > system and can guarantee that you'll always dequeue them fast enough.
> 
> Yes, my system is a hard RT & i want to receive all the interrupts that
> have been generated exactly

No. Linux is not hard RT. You cannot rely on this in a reliable way,
if for any reason the kernel masks interrupt for too long, which can
happen, you'll see that sort of coalescing happening.

If you -really- can't do anything else, then use critical interrupts
but there is little if no support at all for them in linux. It's your
model that is wrong, you should be able to "poll" the device for how
much work (or interrupts) have been sent and react accordingly
regardless of how many actual IRQ triggers came in via the OpenPIC.

> so who is responsible for acking the interrupt? 

At the PIC level, they are acked by the core just before calling into
your handler.

> is there any API in the kernel should i call to do this ack? or 
> it is something Dependant on the device that generates the interrupt?

The PIC-level ack is done for you. I don't know what your DSP does.

> Actaully, the device in my case which is a DSP-core is toggles the
> outpin pin of its GPIO that is connected to the input pin of the
> OpenPIC, so how can 
> i ack this device? do u have any suggestions?

The Ack isn't your problem. Your model is wrong if you design assuming
you will receive all edge interrupts. Being careful about latencies
etc... (and making sure you toggle for long enough, btw, didn't think
about that one, check your MPIC specs) may improve how many of them you
actually receive, -but- you cannot guarantee that you'll get them all,
so even if you somewhat manager into most of your tests to get 100%,
you'll still have an unreliable system.

You must design your communication between the DSP and Linux such that
the interrupt is purely a wakeup call indicating there's work to do, and
some -other- mean for Linux to actually know how much work is to be
done, the actual number of interrupts is not a proper way to do so.

> So how can i achieve this step? how can i ack the interrupt b 4 i call
>  the handler? where can i do this in the kernel?

The kernel does it for you as I said. It's your communication model
that is flawed. Never -ever- rely on edge interrupts in ways that
require them not to be coalesced.

Cheers,
Ben.

> >So you don't need  to worry too much about racing with new incoming
> messages inside the
> > interrupt handler itself. But you need to be prepared to pick up more
> > than one item of work... whatever that is.
> > 
> > Cheers,
> > Ben.
> > 
> > _______________________________________________
> > Linuxppc-dev mailing list
> > Linuxppc-dev@lists.ozlabs.org
> > https://lists.ozlabs.org/listinfo/linuxppc-dev
> > 
> > 
> Quoted from: 
> http://www.nabble.com/Missing-some-interrupts-tp23901807p23906326.html


-- 
View this message in context: http://www.nabble.com/Missing-some-interrupts-tp23901807p23906859.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Missing some interrupts

[wael showair]

wael showair wrote:
> 
>> > You cannot really rely on getting the exact same number of edge
>> > interrupts that were emitted. At least not unless you have a hard RT
>> > system and can guarantee that you'll always dequeue them fast enough.
>> 
>> Yes, my system is a hard RT & i want to receive all the interrupts that
>> have been generated exactly
> 
> No. Linux is not hard RT. You cannot rely on this in a reliable way,
> if for any reason the kernel masks interrupt for too long, which can
> happen, you'll see that sort of coalescing happening.

what about using the RT-Preempt Patch? i can see that it will convert Linux
ti an RTOS.

> 
> If you -really- can't do anything else, then use critical interrupts
> but there is little if no support at all for them in linux. 
>It's your model that is wrong, you should be able to "poll" the device for
how
> much work (or interrupts) have been sent and react accordingly
> regardless of how many actual IRQ triggers came in via the OpenPIC.
> 
>> so who is responsible for acking the interrupt? 
> 
> At the PIC level, they are acked by the core just before calling into
> your handler.
> 

i have check the function of do_IRQ: there is no ack called b 4 calling the
handler but actually it does the following:
1. it gets the irq number using :
       irq = ppc_md.get_irq();

2. then it calls the handler : 
	if (irq != NO_IRQ && irq != NO_IRQ_IGNORE) {
			generic_handle_irq(irq);
	} else if (irq != NO_IRQ_IGNORE)
		/* That's not SMP safe ... but who cares ? */
		ppc_spurious_interrupts++;

i have checked the function of  generic_handle_irq  where it is defined
linux/irq.h & it calls the handler


so from the previous code the processor or the pic does not ack but they
call my ISR handler.
i have also read the specs of my OpenPIC & found that:

"the interrupt handler executing on the processor should then acknowledge
the interrupt by explicitly reading the IACK register.The PIC unit
interprets this read as an interrupt acknowledge (IACK) cycle; in response,
the PIC unit returns the vector associated with the
interrupt source to the interrupt handler routine."

"At the end of the interrupt  the End Of Interrupt (EOI) register must be
set"

So the processor neither reads this iack register nor the handler set the
EOI register.
how is the interrupts works on my board? i dont know?
my linux version is 2.6.27. 

>> is there any API in the kernel should i call to do this ack? or 
>> it is something Dependant on the device that generates the interrupt?
> 
> The PIC-level ack is done for you. I don't know what your DSP does.
> 
>> Actaully, the device in my case which is a DSP-core is toggles the
>> outpin pin of its GPIO that is connected to the input pin of the
>> OpenPIC, so how can 
>> i ack this device? do u have any suggestions?
> 
> The Ack isn't your problem. Your model is wrong if you design assuming
> you will receive all edge interrupts. Being careful about latencies
> etc... (and making sure you toggle for long enough, btw, didn't think
> about that one, check your MPIC specs) may improve how many of them you
> actually receive, -but- you cannot guarantee that you'll get them all,
> so even if you somewhat manager into most of your tests to get 100%,
> you'll still have an unreliable system.
> 
> You must design your communication between the DSP and Linux such that
> the interrupt is purely a wakeup call indicating there's work to do, and
> some -other- mean for Linux to actually know how much work is to be
> done, the actual number of interrupts is not a proper way to do so.
> 
>> So how can i achieve this step? how can i ack the interrupt b 4 i call
>>  the handler? where can i do this in the kernel?
> 
> The kernel does it for you as I said. It's your communication model
> that is flawed. Never -ever- rely on edge interrupts in ways that
> require them not to be coalesced.
> 
> Cheers,
> Ben.
> 
>> >So you don't need  to worry too much about racing with new incoming
>> messages inside the
>> > interrupt handler itself. But you need to be prepared to pick up more
>> > than one item of work... whatever that is.
>> > 
>> > Cheers,
>> > Ben.
>> > 
>> > _______________________________________________
>> > Linuxppc-dev mailing list
>> > Linuxppc-dev@lists.ozlabs.org
>> > https://lists.ozlabs.org/listinfo/linuxppc-dev
>> > 
>> > 
>> Quoted from: 
>> http://www.nabble.com/Missing-some-interrupts-tp23901807p23906326.html
> 
> 
> 
Quoted from: 
http://www.nabble.com/Missing-some-interrupts-tp23901807p23906859.html


-- 
View this message in context: http://www.nabble.com/Missing-some-interrupts-tp23901807p23927820.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Missing some interrupts

[Benjamin Herrenschmidt]

On Mon, 2009-06-08 at 09:45 -0700, wael showair wrote:
> 
> wael showair wrote:
> > 
> >> > You cannot really rely on getting the exact same number of edge
> >> > interrupts that were emitted. At least not unless you have a hard RT
> >> > system and can guarantee that you'll always dequeue them fast enough.
> >> 
> >> Yes, my system is a hard RT & i want to receive all the interrupts that
> >> have been generated exactly
> > 
> > No. Linux is not hard RT. You cannot rely on this in a reliable way,
> > if for any reason the kernel masks interrupt for too long, which can
> > happen, you'll see that sort of coalescing happening.
> 
> what about using the RT-Preempt Patch? i can see that it will convert Linux
> ti an RTOS.

Not only I don't think it's going to guarantee RT response (just improve
it), that's a pretty big hammer to work around what is a design bug in
your system in the first place :-)

> i have check the function of do_IRQ: there is no ack called b 4 calling the
> handler but actually it does the following:

You haven't read properly :-)

> 1. it gets the irq number using :
>        irq = ppc_md.get_irq();
> 
> 2. then it calls the handler : 
> 	if (irq != NO_IRQ && irq != NO_IRQ_IGNORE) {
> 			generic_handle_irq(irq);
> 	} else if (irq != NO_IRQ_IGNORE)
> 		/* That's not SMP safe ... but who cares ? */
> 		ppc_spurious_interrupts++;
> 
> i have checked the function of  generic_handle_irq  where it is defined
> linux/irq.h & it calls the handler

No. It call the -flow- handler, which is a different thing (yes, the
terminology can be a bit confusing). The flow handler is configured
by the interrupt controller itself and will do whatever necessary
ack'ing, masking, EOI'ing etc... for a given PIC around the call to the
actual driver handler.

> so from the previous code the processor or the pic does not ack but they
> call my ISR handler.

They do. You missed it.

> i have also read the specs of my OpenPIC & found that:
> 
> "the interrupt handler executing on the processor should then acknowledge
> the interrupt by explicitly reading the IACK register.

Reading the IACK is what get_irq() does on OpenPIC.

> The PIC unit
> interprets this read as an interrupt acknowledge (IACK) cycle; in response,
> the PIC unit returns the vector associated with the
> interrupt source to the interrupt handler routine."
> 
> "At the end of the interrupt  the End Of Interrupt (EOI) register must be
> set"

Which is done by the flow handler.

> So the processor neither reads this iack register nor the handler set the
> EOI register.
 
You obviously missed both :-)

Ben.