[Xenomai-core] Error propagating ISR to Linux domain

[Xenomai-core] Error propagating ISR to Linux domain

[Benjamin ZORES]

Hi,

I've encountered a little problem when trying to propagate an ISR from 
Xenomai to Linux context.

To sum up clearly, I'm writing an RT kernel module that drives a PCI card.
I've registered a nucleus ISR handler on the IRQ attributed to this card.

On some systems, this IRQ is dedicated to this PCI card only.
Though, on others, it is shared with other peripherals.
Hence, I need to forward/propagate this IRQ to Linux domain for other 
drivers to handle it
when the IRQ has not been initiated by my PCI card.

My RT ISR code looks like the following:

int my_isr_handler (xnintr_t *irq)
{
    ... /* handling */
   return XN_ISR_HANDLED | XN_ISR_PROPAGATE;
}

While this code works perfectly when the IRQ number is really shared 
among peripherals,
when I use it on system where this IRQ is dedicated to RT only (i.e. not 
used by Linux),
then, the IRQ is never acknowledged.

In other words, I see a bug when returning XN_ISR_PROPAGATE on a system 
where
no Linux driver has registered a handler for this IRQ.
Is this something known and is there some workaround (I'm using 2.6.23 + 
lastest Adeos + Xenomai 2.4.4 on x86_32) ???

A quick and dirty workaround is to create a dummy Linux IRQ handler 
(doing so acknowledge my propagated IRQ) but:
 - if it returns IRQ_NONE, then Linux discard the IRQ line after having 
missed 100000 of them (on system where the IRQ is dedicated)
 - if it returns IRQ_HANDLED, then I have no guarantee that this dumym 
handler won't get called before the legacy drivers that need it.

Or maybe there is a way in Xenomai to know that any subdomain (like 
Linux) has registered a handler for this given IRQ ??

Any help would be appreciated,

Ben

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Jan Kiszka]

Benjamin ZORES wrote:
> Hi,
> 
> I've encountered a little problem when trying to propagate an ISR from 
> Xenomai to Linux context.
> 
> To sum up clearly, I'm writing an RT kernel module that drives a PCI card.
> I've registered a nucleus ISR handler on the IRQ attributed to this card.
> 
> On some systems, this IRQ is dedicated to this PCI card only.
> Though, on others, it is shared with other peripherals.
> Hence, I need to forward/propagate this IRQ to Linux domain for other 
> drivers to handle it
> when the IRQ has not been initiated by my PCI card.

If you also need deterministic IRQ handling in the shared case, that
approach won't fly. It may work to some degree if there are significant
gaps between the individual IRQs, so significant that one can assume
Linux will always be reactive enough to handle the last forwarded IRQ
(and thus release the line before the RT event arrives).

The only truly deterministic solution is to write minimal IRQ handlers
for the involved Linux devices in RT space, ie. for Xenomai. RT-IRQ
sharing works nicely.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Ulrich Schwab]

why not checking for irq origin like this:
int my_isr_handler (xnintr_t *irq)
{
 if ( ! test_my_card_for_irq_origin )
    return XN_ISR_NONE | XN_ISR_PROPAGATE;
  ... /* handling */
 return XN_ISR_HANDLED;
}

this way XN_ISR_PROPAGATE is never returned in the not-shared case.

On Wed, Jul 16, 2008 at 6:03 PM, Benjamin ZORES
<benjamin.zores@domain.hid> wrote:
> Hi,
>
> I've encountered a little problem when trying to propagate an ISR from
> Xenomai to Linux context.
>
> To sum up clearly, I'm writing an RT kernel module that drives a PCI card.
> I've registered a nucleus ISR handler on the IRQ attributed to this card.
>
> On some systems, this IRQ is dedicated to this PCI card only.
> Though, on others, it is shared with other peripherals.
> Hence, I need to forward/propagate this IRQ to Linux domain for other
> drivers to handle it
> when the IRQ has not been initiated by my PCI card.
>
> My RT ISR code looks like the following:
>
> int my_isr_handler (xnintr_t *irq)
> {
>    ... /* handling */
>   return XN_ISR_HANDLED | XN_ISR_PROPAGATE;
> }
>
> While this code works perfectly when the IRQ number is really shared
> among peripherals,
> when I use it on system where this IRQ is dedicated to RT only (i.e. not
> used by Linux),
> then, the IRQ is never acknowledged.
>
> In other words, I see a bug when returning XN_ISR_PROPAGATE on a system
> where
> no Linux driver has registered a handler for this IRQ.
> Is this something known and is there some workaround (I'm using 2.6.23 +
> lastest Adeos + Xenomai 2.4.4 on x86_32) ???
>
> A quick and dirty workaround is to create a dummy Linux IRQ handler
> (doing so acknowledge my propagated IRQ) but:
>  - if it returns IRQ_NONE, then Linux discard the IRQ line after having
> missed 100000 of them (on system where the IRQ is dedicated)
>  - if it returns IRQ_HANDLED, then I have no guarantee that this dumym
> handler won't get called before the legacy drivers that need it.
>
> Or maybe there is a way in Xenomai to know that any subdomain (like
> Linux) has registered a handler for this given IRQ ??
>
> Any help would be appreciated,
>
> Ben
>
> _______________________________________________
> Xenomai-core mailing list
> Xenomai-core@domain.hid
> https://mail.gna.org/listinfo/xenomai-core
>

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Gilles Chanteperdrix]

Ulrich Schwab wrote:
> why not checking for irq origin like this:
> int my_isr_handler (xnintr_t *irq)
> {
>  if ( ! test_my_card_for_irq_origin )
>     return XN_ISR_NONE | XN_ISR_PROPAGATE;
>   ... /* handling */
>  return XN_ISR_HANDLED;
> }
> 
> this way XN_ISR_PROPAGATE is never returned in the not-shared case.

I think this idea needs an answer; the answer is no: it will not work.
Because the IRQ will remain masked until Linux handles it, which
basically means that the RT irq will wait for non-RT activity, you loose
real-time response.

The only approach that works is, detailing a bit more what Jan
suggested, assuming that driver1 is RT and driver2 is non RT:

int driver2_nrt_irq_pending;

int driver2_rt_isr_handler(xnintr_t *irq)
{
   if (!test_driver2_hard_irq_pending())
 	return XN_ISR_NONE;

   clear_driver2_hard_irq();
   driver2_nrt_irq_pending = 1;

   return XN_ISR_HANDLED | XN_ISR_PROPAGATE;
}

int driver1_rt_isr_handler(xnintr_t *irq)
{
   if (!test_driver1_irq_pending())
	return XN_ISR_NONE;

   /* driver1 handling */

   return XN_ISR_HANDLED;
}

int driver2_nrt_isr_handler(int irq, void *dev_id)
{
#if 0
	/* The old code checking and clearing hardware irqs. */
        if (!test_drive2_hard_irq_pending())
		return IRQ_NONE;

	clear_drive2_hard_irq();
#else
	/* Replaced by this code. */
	if (!driver2_nrt_irq_pending)
		return IRQ_NONE;

	driver2_nrt_irq_pending = 0;
#endif

	/* driver2 irq handling. */

	return IRQ_HANDLED;
}

-- 
                                                 Gilles.

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> Ulrich Schwab wrote:
>> why not checking for irq origin like this:
>> int my_isr_handler (xnintr_t *irq)
>> {
>>  if ( ! test_my_card_for_irq_origin )
>>     return XN_ISR_NONE | XN_ISR_PROPAGATE;
>>   ... /* handling */
>>  return XN_ISR_HANDLED;
>> }
>>
>> this way XN_ISR_PROPAGATE is never returned in the not-shared case.
> 
> I think this idea needs an answer; the answer is no: it will not work.
> Because the IRQ will remain masked until Linux handles it, which
> basically means that the RT irq will wait for non-RT activity, you loose
> real-time response.

It will not work, but not for the reason I mention: Adeos WILL re-enable
the IRQ at interrupt controller level after the end of this handler,
however, since the interrupt was not cleared on the peripheral side, the
peripheral will reassert the interrupt when the interrupt is unmasked at
interrupt controller level, so the CPU will enter an infinite loop
invoking the ISR.

-- 
                                                 Gilles.

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Ulrich Schwab]

On Tue, Jul 29, 2008 at 4:20 PM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> Gilles Chanteperdrix wrote:
>> Ulrich Schwab wrote:
>>> why not checking for irq origin like this:
>>> int my_isr_handler (xnintr_t *irq)
>>> {
>>>  if ( ! test_my_card_for_irq_origin )
>>>     return XN_ISR_NONE | XN_ISR_PROPAGATE;
>>>   ... /* handling */
>>>  return XN_ISR_HANDLED;
>>> }
>>>
>>> this way XN_ISR_PROPAGATE is never returned in the not-shared case.
>>
>> I think this idea needs an answer; the answer is no: it will not work.
>> Because the IRQ will remain masked until Linux handles it, which
>> basically means that the RT irq will wait for non-RT activity, you loose
>> real-time response.
>
> It will not work, but not for the reason I mention: Adeos WILL re-enable
> the IRQ at interrupt controller level after the end of this handler,
> however, since the interrupt was not cleared on the peripheral side, the
> peripheral will reassert the interrupt when the interrupt is unmasked at
> interrupt controller level, so the CPU will enter an infinite loop
> invoking the ISR.
>
> --
>                                                 Gilles.
>
Are You sure?
IIRC the infinte loop You describe occurs when XN_ISR_PROPAGATE is
removed from the code above.


Ulrich

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Gilles Chanteperdrix]

Ulrich Schwab wrote:
> On Tue, Jul 29, 2008 at 4:20 PM, Gilles Chanteperdrix
> <gilles.chanteperdrix@xenomai.org> wrote:
>> Gilles Chanteperdrix wrote:
>>> Ulrich Schwab wrote:
>>>> why not checking for irq origin like this:
>>>> int my_isr_handler (xnintr_t *irq)
>>>> {
>>>>  if ( ! test_my_card_for_irq_origin )
>>>>     return XN_ISR_NONE | XN_ISR_PROPAGATE;
>>>>   ... /* handling */
>>>>  return XN_ISR_HANDLED;
>>>> }
>>>>
>>>> this way XN_ISR_PROPAGATE is never returned in the not-shared case.
>>> I think this idea needs an answer; the answer is no: it will not work.
>>> Because the IRQ will remain masked until Linux handles it, which
>>> basically means that the RT irq will wait for non-RT activity, you loose
>>> real-time response.
>> It will not work, but not for the reason I mention: Adeos WILL re-enable
>> the IRQ at interrupt controller level after the end of this handler,
>> however, since the interrupt was not cleared on the peripheral side, the
>> peripheral will reassert the interrupt when the interrupt is unmasked at
>> interrupt controller level, so the CPU will enter an infinite loop
>> invoking the ISR.
>>
>> --
>>                                                 Gilles.
>>
> Are You sure?
> IIRC the infinte loop You describe occurs when XN_ISR_PROPAGATE is
> removed from the code above.

Yes, I am sure, the only solution that works is the one I posted.

-- 


					    Gilles.

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Gilles Chanteperdrix]

Ulrich Schwab wrote:
> On Tue, Jul 29, 2008 at 4:20 PM, Gilles Chanteperdrix
> <gilles.chanteperdrix@xenomai.org> wrote:
>> Gilles Chanteperdrix wrote:
>>> Ulrich Schwab wrote:
>>>> why not checking for irq origin like this:
>>>> int my_isr_handler (xnintr_t *irq)
>>>> {
>>>>  if ( ! test_my_card_for_irq_origin )
>>>>     return XN_ISR_NONE | XN_ISR_PROPAGATE;
>>>>   ... /* handling */
>>>>  return XN_ISR_HANDLED;
>>>> }
>>>>
>>>> this way XN_ISR_PROPAGATE is never returned in the not-shared case.
>>> I think this idea needs an answer; the answer is no: it will not work.
>>> Because the IRQ will remain masked until Linux handles it, which
>>> basically means that the RT irq will wait for non-RT activity, you loose
>>> real-time response.
>> It will not work, but not for the reason I mention: Adeos WILL re-enable
>> the IRQ at interrupt controller level after the end of this handler,
>> however, since the interrupt was not cleared on the peripheral side, the
>> peripheral will reassert the interrupt when the interrupt is unmasked at
>> interrupt controller level, so the CPU will enter an infinite loop
>> invoking the ISR.
>>
>> --
>>                                                 Gilles.
>>
> Are You sure?
> IIRC the infinte loop You describe occurs when XN_ISR_PROPAGATE is
> removed from the code above.

Actually, no, I was not sure, so, I had a look at the code, all IRQs
handlers do:

        if (s & XN_ISR_PROPAGATE)
                xnarch_chain_irq(irq);
        else if (!(s & XN_ISR_NOENABLE))
                xnarch_end_irq(irq);

It means that when XN_ISR_PROPAGATE is set the IRQ does not get unmasked
at interrupt controller level, and your solution does not cause the
system to enter the infinite loop, however you loose the RT response, as
was (correctly) said in my first answer.

It also means that my solution has the same issue. So, we should add a
call to xnarch_end_irq in the non-rt driver RT stub. So, the solution
becomes:

int driver2_nrt_irq_pending;

int driver2_rt_isr_handler(xnintr_t *irq)
{
   if (!test_driver2_hard_irq_pending())
 	return XN_ISR_NONE;

   clear_driver2_hard_irq();
   driver2_nrt_irq_pending = 1;

   xnarch_end_irq(irq->irq);

   return XN_ISR_HANDLED | XN_ISR_PROPAGATE;
}

int driver1_rt_isr_handler(xnintr_t *irq)
{
   if (!test_driver1_irq_pending())
	return XN_ISR_NONE;

   /* driver1 handling */

   return XN_ISR_HANDLED;
}

int driver2_nrt_isr_handler(int irq, void *dev_id)
{
#ifndef CONFIG_IPIPE
	/* The old code checking and clearing hardware irqs. */
        if (!test_drive2_hard_irq_pending())
		return IRQ_NONE;

	clear_drive2_hard_irq();
#else /* IPIPE */
	/* Replaced by this code. */
	if (!driver2_nrt_irq_pending)
		return IRQ_NONE;

	driver2_nrt_irq_pending = 0;
#endif /* IPIPE */

	/* driver2 irq handling. */

	return IRQ_HANDLED;
}

-- 
                                                 Gilles.

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Ulrich Schwab]

On Wed, Jul 30, 2008 at 10:31 AM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> ...  you loose the RT response, as was (correctly) said in my first answer.
You are right, of course.
It was my error not to mention this in the original post.

For me this is just the solution to avoid the infinte loop when irqs are shared.
Until now it was always possible to avoid shared irqs (between RT and non-RT)
by changing the position of the PCI card.

Ulrich

Re: [Xenomai-core] Error propagating ISR to Linux domain

[Gilles Chanteperdrix]

Ulrich Schwab wrote:
> On Wed, Jul 30, 2008 at 10:31 AM, Gilles Chanteperdrix
> <gilles.chanteperdrix@xenomai.org> wrote:
>> ...  you loose the RT response, as was (correctly) said in my first answer.
> You are right, of course.
> It was my error not to mention this in the original post.
> 
> For me this is just the solution to avoid the infinte loop when irqs are shared.
> Until now it was always possible to avoid shared irqs (between RT and non-RT)
> by changing the position of the PCI card.

It looks like some people can not avoid the shared IRQ between RT and
non RT. It is amazing that with APIC/IO-APIC PCs have something like 255
irqs, but that the PCs still manage to put several peripherals on the
same IRQ.

-- 
                                                 Gilles.