Re: IRQ timeout

[John Ogness <john.ogness@linutronix.de>]

On 2009-09-08, Hans-Peter Bock <Hans-Peter.Bock@isw.uni-stuttgart.de> wrote:
> when I start my application using the uio_sercos3 driver, the network on my e100 stops working after a short time. Some miutes later I got this message in the console and my network started working again:
>
> [  725.000141] ------------[ cut here ]------------
> [  725.000185] WARNING: at net/sched/sch_generic.c:230 dev_watchdog+0xf5/0x186()
> [  725.000226] Hardware name: GA-73PVM-S2H
> [  725.000262] NETDEV WATCHDOG: eth1 (e100): transmit timed out
> [  725.000299] Modules linked in: uio_sercos3 uio ipv6 sbp2 loop com20020_pci serio_raw com20020 e100 ohci1394 ohci_hcd amd74xx ehci_hcd arcnet parport_pc psmouse mii ieee1394 pcspkr usbcore ide_pci_generic parport evdev
> [  725.000881] Pid: 5, comm: sirq-timer/0 Not tainted 2.6.29.5-rt22 #3
> [  725.000919] Call Trace:
> [  725.000957]  [<c012b571>] warn_slowpath+0x71/0xa8
> [  725.000996]  [<c011d5fc>] ? pick_next_highest_task_rt+0xc6/0xd8
> [  725.001039]  [<c011f3b1>] ? enqueue_task_rt+0x13a/0x232
> [  725.001100]  [<c0348e00>] ? __read_trylock+0x3e/0x47
> [  725.001139]  [<c0172136>] ? cpupri_set+0xdd/0xfb
> [  725.001177]  [<c0162bed>] ? __rcu_read_unlock+0x69/0x78
> [  725.001215]  [<c0348d5f>] ? __spin_unlock+0xf/0x23
> [  725.001253]  [<c0174326>] ? perf_counter_task_sched_in+0x14e/0x15c
> [  725.001293]  [<c021f8c2>] ? strlcpy+0x17/0x49
> [  725.001331]  [<c02f398e>] dev_watchdog+0xf5/0x186
> [  725.001369]  [<c0144fa1>] ? clocksource_watchdog+0x22f/0x23a
> [  725.001408]  [<c0122d50>] ? __wake_up+0x28/0x32
> [  725.001445]  [<c02f3899>] ? dev_watchdog+0x0/0x186
> [  725.001484]  [<c013370c>] run_timer_softirq+0x19a/0x22a
> [  725.001522]  [<c02f3899>] ? dev_watchdog+0x0/0x186
> [  725.001560]  [<c012fb5a>] ksoftirqd+0x134/0x228
> [  725.001597]  [<c012fa26>] ? ksoftirqd+0x0/0x228
> [  725.001635]  [<c013c399>] kthread+0x3b/0x61
> [  725.001672]  [<c013c35e>] ? kthread+0x0/0x61
> [  725.001710]  [<c010350f>] kernel_thread_helper+0x7/0x10
> [  725.001747] ---[ end trace 9b182b0252a7e099 ]---
> [  725.014615] e100 0000:01:06.0: firmware: requesting e100/d101m_ucode.bin
> [  725.029213] e100: eth1 NIC Link is Up 100 Mbps Full Duplex
>
> What can I do to solve this issue?

The following information would be helpful:

1. Is the uio_sercos3 driver sharing its interrupt?
   $ cat /proc/interrupts

2. What are the rt-priorities for your application (including the UIO
   thread) and for the network stack threads?
   $ ps -Alm

3. How many interrupts per second (approx.) are being generated by the
   Sercos hardware while your application is running?
   $ cat /proc/interrupts; sleep 1; cat /proc/interrupts

4. What is the system load while your application is running and when
   the network stops working?
   $ cat /proc/loadavg

5. Is your application doing anything particular with the network (such
   as tunneling Sercos data)?

John Ogness