[Xenomai] "watchdog triggered" error & irq interrupt

[Xenomai] "watchdog triggered" error & irq interrupt

[Ran Shalit]

Hello,

On testing interrupt within kernel it seems to work ok.
I then moved to userspace rt thread waiting on events coming from the
rtfm kernel driver (witch signals with rtd_event_signal)

But then I get the following errors:

dma0chan1-copy0: #15195: got completion callback, but status is 'in progress'
dma0chan3-copy0: #14335: test timed out
dma0chan3-copy0: #14336: got completion callback, but status is 'in progress'
Xenomai: watchdog triggered -- signaling runaway thread 'rtdm'
[sched_delayed] sched: RT throttling activated
Xenomai: RTDM: closing file descriptor 0.
CPU time limit exceeded

I understand that it mean that cpu is probably in busy wait.
The interrupt is done on raising edge, and I configure it as following:
rtdm driver:
========
ret = rtdm_irq_request(&ctx->irq_handle,
 91, PFI_IRQHandler, RTDM_IRQTYPE_EDGE,
 DRVNAM, ctx);

userspace real-time thread
===============

while (1) {
    rt_printf("11\n");
    if (ioctl(file_desc, RTTST_RTIOC_GPIOIRQ_WAIT_IRQ))
    {
           rt_printf("failed!");
     };
     rt_printf("22\n");
      mymmap[0xe4] = 1;  //mymmap is memory mapped io to 0x80010000
      writel(1, rtdm_fpga_base_addr + 0xe4);
      g_value[g_index] = readl( rtdm_fpga_base_addr+ 0x8010);
     pCnt = &(ledRegister[0x8010]);
      mymmap[0xe4] = 0;

   }

I've been debugging it for some hours in vain. I have no idea, any
suggestion will help.

complete sources (driver & app)
https://drive.google.com/folderview?id=0B22GsWueReZTS3RaV3gzRk9aZzQ&usp=sharing

Regards,
Ran

Re: [Xenomai] "watchdog triggered" error & irq interrupt

[Gilles Chanteperdrix]

On Thu, May 26, 2016 at 06:57:49PM +0300, Ran Shalit wrote:
> Hello,
> 
> On testing interrupt within kernel it seems to work ok.
> I then moved to userspace rt thread waiting on events coming from the
> rtfm kernel driver (witch signals with rtd_event_signal)
> 
> But then I get the following errors:
> 
> dma0chan1-copy0: #15195: got completion callback, but status is 'in progress'
> dma0chan3-copy0: #14335: test timed out
> dma0chan3-copy0: #14336: got completion callback, but status is 'in progress'
> Xenomai: watchdog triggered -- signaling runaway thread 'rtdm'
> [sched_delayed] sched: RT throttling activated
> Xenomai: RTDM: closing file descriptor 0.
> CPU time limit exceeded
> 
> I understand that it mean that cpu is probably in busy wait.
> The interrupt is done on raising edge, and I configure it as following:
> rtdm driver:
> ========
> ret = rtdm_irq_request(&ctx->irq_handle,
>  91, PFI_IRQHandler, RTDM_IRQTYPE_EDGE,
>  DRVNAM, ctx);
> 
> userspace real-time thread
> ===============
> 
> while (1) {
>     rt_printf("11\n");
>     if (ioctl(file_desc, RTTST_RTIOC_GPIOIRQ_WAIT_IRQ))
>     {
>            rt_printf("failed!");
>      };
>      rt_printf("22\n");

If what follows is what is necessary to acknowledge the irq at the
device level, it has to be done in the interrupt handler. Otherwise
when the interrupt handler has finished executing, interrupts are
enabled, and the interrupt triggers again, leaving no chance to the
interrupt handler to run. Alternatively, you may want to disable the
interrupt line in the interrupt handler and reenable it after the
thread has run. Note however that it increases the time necessary to
handle the interrupt and may result in lost interrupts (if they
happen while the line is masked).

>       mymmap[0xe4] = 1;  //mymmap is memory mapped io to 0x80010000
>       writel(1, rtdm_fpga_base_addr + 0xe4);
>       g_value[g_index] = readl( rtdm_fpga_base_addr+ 0x8010);
>      pCnt = &(ledRegister[0x8010]);
>       mymmap[0xe4] = 0;
> 
>    }
> 
> I've been debugging it for some hours in vain. I have no idea, any
> suggestion will help.
> 
> complete sources (driver & app)
> https://drive.google.com/folderview?id=0B22GsWueReZTS3RaV3gzRk9aZzQ&usp=sharing

This code is crap. If you want people to review your code, please
make it short and remove the cruft that you do not use, and remove
the dead code too. As I already told you, you should start from the
"skeleton" code, not from the RTDM test driver which is a unit test
for the RTDM API.

-- 
					    Gilles.
https://click-hack.org

Re: [Xenomai] "watchdog triggered" error & irq interrupt

[Ran Shalit]

On Thu, May 26, 2016 at 7:28 PM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> On Thu, May 26, 2016 at 06:57:49PM +0300, Ran Shalit wrote:
>> Hello,
>>
>> On testing interrupt within kernel it seems to work ok.
>> I then moved to userspace rt thread waiting on events coming from the
>> rtfm kernel driver (witch signals with rtd_event_signal)
>>
>> But then I get the following errors:
>>
>> dma0chan1-copy0: #15195: got completion callback, but status is 'in progress'
>> dma0chan3-copy0: #14335: test timed out
>> dma0chan3-copy0: #14336: got completion callback, but status is 'in progress'
>> Xenomai: watchdog triggered -- signaling runaway thread 'rtdm'
>> [sched_delayed] sched: RT throttling activated
>> Xenomai: RTDM: closing file descriptor 0.
>> CPU time limit exceeded
>>
>> I understand that it mean that cpu is probably in busy wait.
>> The interrupt is done on raising edge, and I configure it as following:
>> rtdm driver:
>> ========
>> ret = rtdm_irq_request(&ctx->irq_handle,
>>  91, PFI_IRQHandler, RTDM_IRQTYPE_EDGE,
>>  DRVNAM, ctx);
>>
>> userspace real-time thread
>> ===============
>>
>> while (1) {
>>     rt_printf("11\n");
>>     if (ioctl(file_desc, RTTST_RTIOC_GPIOIRQ_WAIT_IRQ))
>>     {
>>            rt_printf("failed!");
>>      };
>>      rt_printf("22\n");
>
> If what follows is what is necessary to acknowledge the irq at the
> device level, it has to be done in the interrupt handler. Otherwise
> when the interrupt handler has finished executing, interrupts are
> enabled, and the interrupt triggers again, leaving no chance to the
> interrupt handler to run. Alternatively, you may want to disable the
> interrupt line in the interrupt handler and reenable it after the
> thread has run. Note however that it increases the time necessary to
> handle the interrupt and may result in lost interrupts (if they
> happen while the line is masked).

That followed code is not for acknoweledging the irq but just for
reading timer ticks.
The irq handler in rtdm driver handles the raising edge:

static int PFI_IRQHandler(rtdm_irq_t *irq_handle)
{
struct rtdm_test_context *ctx;

ctx = rtdm_irq_get_arg(irq_handle, struct rtdm_test_context);
  rtdm_event_signal(&ctx->irq_event);
   return RTDM_IRQ_HANDLED;
}

I suppose that cpu (zynq) should
disable the interrupt becuase it is defined as raising edge in device tree:

fpga_device_tree@80000000 {
compatible = "xillybus,xillybus_lite_of-1.00.a";
reg = < 0x80000000 0x400000 >;
interrupts = < 0 59 1 >;
interrupt-parent = <&ps7_scugic_0>;
} ;

How can I be sure that the issue is that the interrupts are not acknowledged ?




>
>>       mymmap[0xe4] = 1;  //mymmap is memory mapped io to 0x80010000
>>       writel(1, rtdm_fpga_base_addr + 0xe4);
>>       g_value[g_index] = readl( rtdm_fpga_base_addr+ 0x8010);
>>      pCnt = &(ledRegister[0x8010]);
>>       mymmap[0xe4] = 0;
>>
>>    }
>>
>> I've been debugging it for some hours in vain. I have no idea, any
>> suggestion will help.
>>
>> complete sources (driver & app)
>> https://drive.google.com/folderview?id=0B22GsWueReZTS3RaV3gzRk9aZzQ&usp=sharing
>
> This code is crap. If you want people to review your code, please
> make it short and remove the cruft that you do not use, and remove
> the dead code too. As I already told you, you should start from the
> "skeleton" code, not from the RTDM test driver which is a unit test
> for the RTDM API.

That's Right...
I made many many tests moving from one attitude to another, failing in
all this trials.
I've also tried to use interrupts in userspace (user_irq.c example),
but could not get any interrupt with that method, so I moved back to
rtdm driver, although doing it all in userspace, if it worked, could
be better.

Thank you,
Ran

> --
>                                             Gilles.
> https://click-hack.org

Re: [Xenomai] "watchdog triggered" error & irq interrupt

[Gilles Chanteperdrix]

On Thu, May 26, 2016 at 07:52:20PM +0300, Ran Shalit wrote:
> On Thu, May 26, 2016 at 7:28 PM, Gilles Chanteperdrix
> <gilles.chanteperdrix@xenomai.org> wrote:
> > On Thu, May 26, 2016 at 06:57:49PM +0300, Ran Shalit wrote:
> >> Hello,
> >>
> >> On testing interrupt within kernel it seems to work ok.
> >> I then moved to userspace rt thread waiting on events coming from the
> >> rtfm kernel driver (witch signals with rtd_event_signal)
> >>
> >> But then I get the following errors:
> >>
> >> dma0chan1-copy0: #15195: got completion callback, but status is 'in progress'
> >> dma0chan3-copy0: #14335: test timed out
> >> dma0chan3-copy0: #14336: got completion callback, but status is 'in progress'
> >> Xenomai: watchdog triggered -- signaling runaway thread 'rtdm'
> >> [sched_delayed] sched: RT throttling activated
> >> Xenomai: RTDM: closing file descriptor 0.
> >> CPU time limit exceeded
> >>
> >> I understand that it mean that cpu is probably in busy wait.
> >> The interrupt is done on raising edge, and I configure it as following:
> >> rtdm driver:
> >> ========
> >> ret = rtdm_irq_request(&ctx->irq_handle,
> >>  91, PFI_IRQHandler, RTDM_IRQTYPE_EDGE,
> >>  DRVNAM, ctx);
> >>
> >> userspace real-time thread
> >> ===============
> >>
> >> while (1) {
> >>     rt_printf("11\n");
> >>     if (ioctl(file_desc, RTTST_RTIOC_GPIOIRQ_WAIT_IRQ))
> >>     {
> >>            rt_printf("failed!");

You should break from the loop here, otherwise if the ioctl
constantly fails, you will have an endless loop (and the message
will never be printed).

> >>      };
> >>      rt_printf("22\n");
> >
> > If what follows is what is necessary to acknowledge the irq at the
> > device level, it has to be done in the interrupt handler. Otherwise
> > when the interrupt handler has finished executing, interrupts are
> > enabled, and the interrupt triggers again, leaving no chance to the
> > interrupt handler to run. Alternatively, you may want to disable the
> > interrupt line in the interrupt handler and reenable it after the
> > thread has run. Note however that it increases the time necessary to
> > handle the interrupt and may result in lost interrupts (if they
> > happen while the line is masked).
> 
> That followed code is not for acknoweledging the irq but just for
> reading timer ticks.

Well, Xenomai provides functions for that which will more clearly
indicate your intent to those reading your code.

> The irq handler in rtdm driver handles the raising edge:
> 
> static int PFI_IRQHandler(rtdm_irq_t *irq_handle)
> {
> struct rtdm_test_context *ctx;
> 
> ctx = rtdm_irq_get_arg(irq_handle, struct rtdm_test_context);
>   rtdm_event_signal(&ctx->irq_event);
>    return RTDM_IRQ_HANDLED;
> }
> 
> I suppose that cpu (zynq) should
> disable the interrupt becuase it is defined as raising edge in device tree:
> 
> fpga_device_tree@80000000 {
> compatible = "xillybus,xillybus_lite_of-1.00.a";
> reg = < 0x80000000 0x400000 >;
> interrupts = < 0 59 1 >;
> interrupt-parent = <&ps7_scugic_0>;
> } ;
> 
> How can I be sure that the issue is that the interrupts are not acknowledged ?

RTDM itself does not take the device tree settings into account. So,
if Linux does not do it if you do not call request_irq, nobody took
these settings into account. In order to be sure, I would check the
interrupt controller registers to see if it is correctly configured
in rising edge mode.

> >> https://drive.google.com/folderview?id=0B22GsWueReZTS3RaV3gzRk9aZzQ&usp=sharing
> >
> > This code is crap. If you want people to review your code, please
> > make it short and remove the cruft that you do not use, and remove
> > the dead code too. As I already told you, you should start from the
> > "skeleton" code, not from the RTDM test driver which is a unit test
> > for the RTDM API.
> 
> That's Right...
> I made many many tests moving from one attitude to another, failing in
> all this trials.
> I've also tried to use interrupts in userspace (user_irq.c
> example),

Don't. Handling interrupts in user-space is a bad idea.

> but could not get any interrupt with that method, so I moved back to
> rtdm driver, although doing it all in userspace, if it worked, could
> be better.

I do not really care, if you want me to read your code, send short
and readable code.

-- 
					    Gilles.
https://click-hack.org