high prioritized threaded interrupt is delayed due to low priority softirq job

yosi yarchi
2022-11-17T06:55:18Z

Hi

my platform is:
SAM9X35 (8 CAN mailboxes),
Linux mscb 5.4.41-linux4sam-2020.04-rt24 #1 PREEMPT_RT Wed Nov 9
06:12:28 UTC 2022 armv5tejl GNU/Linux
CAN threaded interrupt (irq/30-can0-346) is configured to be highest
priority on
system, and it is scheduled very quickly after CAN irq_handler expires
(irq_handler_entry: irq=30 name=can0), doing its job (at91_irq).

my problem is that from time to time I get CAN RX overflow. problem
analysis is as the following (see trace below):
after CAN threaded interrupt completes its job (at91_irq_r),
scheduler force it to
run some other low priority softirq jobs (softirq_entry: vec=1
[action=TIMER],
softirq_entry: vec=3 [action=NET_RX]). during the processing of
those jobs, new HW
CAN irq_handler expires (irq_handler_entry: irq=30 name=can0), but
scheduler
schedule the at91_irq job at the end of job queue, which is too
late, and I get CAN
rx overflow.

I've tried some solutions, no one worked out of the box:
1. disable all relevant irqs (timer, network) at CAN irq_handler
(irq_handler_entry:
irq=30 name=can0), and enable them at the end of CAN threaded
interrupt job
processing (at91_irq_r)
2. configure CAN job (at91_irq) to run inside the CAN irq_handler
(irq_handler_entry:
irq=30 name=can0) by adding IRQF_NO_THREAD flag to request_irq.

is there built-in solution for this problem at linux-rt?
if not, what could be good 'hacked' solutions for the problem?

with best regards
Yosi Yarchi

kernel trace:
-------------

<idle>-0     [000] d..h1..  6020.481859: irq_handler_entry:
irq=30 name=can0
<idle>-0     [000] d..h1..  6020.481875: irq_handler_exit:
irq=30 ret=handled
irq/30-can0-346   [000] d......  6020.481972: at91_irq:
(at91_irq+0x0/0xab0)
irq/30-can0-346   [000] d...1..  6020.482090: at91_irq_r:
(irq_forced_thread_fn+0x3c/0x8c <- at91_irq)
<idle>-0     [000] d..h1..  6020.482330: irq_handler_entry:
irq=30 name=can0
<idle>-0     [000] d..h1..  6020.482347: irq_handler_exit:
irq=30 ret=handled
irq/30-can0-346   [000] d......  6020.482438: at91_irq:
(at91_irq+0x0/0xab0)
irq/30-can0-346   [000] d...1..  6020.482553: at91_irq_r:
(irq_forced_thread_fn+0x3c/0x8c <- at91_irq)
<idle>-0     [000] d..h1..  6020.482812: irq_handler_entry:
irq=30 name=can0
<idle>-0     [000] d..h1..  6020.482829: irq_handler_exit:
irq=30 ret=handled
irq/30-can0-346   [000] d......  6020.482923: at91_irq:
(at91_irq+0x0/0xab0)
irq/30-can0-346   [000] d...1..  6020.483040: at91_irq_r:
(irq_forced_thread_fn+0x3c/0x8c <- at91_irq)
<idle>-0     [000] d..h1..  6020.483290: irq_handler_entry:
irq=30 name=can0
<idle>-0     [000] d..h1..  6020.483308: irq_handler_exit:
irq=30 ret=handled
irq/30-can0-346   [000] d......  6020.483400: at91_irq:
(at91_irq+0x0/0xab0)
irq/30-can0-346   [000] d...1..  6020.483514: at91_irq_r:
(irq_forced_thread_fn+0x3c/0x8c <- at91_irq)
<idle>-0     [000] d..h1..  6020.483781: irq_handler_entry:
irq=30 name=can0
<idle>-0     [000] d..h1..  6020.483799: irq_handler_exit:
irq=30 ret=handled
irq/30-can0-346   [000] d..h...  6020.483891: irq_handler_entry:
irq=16 name=tc_clkevt
irq/30-can0-346   [000] d..h...  6020.483956: softirq_raise: vec=1
[action=TIMER]
irq/30-can0-346   [000] d..h...  6020.484006: irq_handler_exit:
irq=16 ret=handled
irq/30-can0-346   [000] d......  6020.484048: at91_irq:
(at91_irq+0x0/0xab0)
irq/30-can0-346   [000] d...1..  6020.484169: at91_irq_r:
(irq_forced_thread_fn+0x3c/0x8c <- at91_irq)
irq/30-can0-346   [000] .......  6020.484207: softirq_entry: vec=1
[action=TIMER]
irq/30-can0-346   [000] d..h1..  6020.484254: irq_handler_entry:
irq=30 name=can0
irq/30-can0-346   [000] d..h1..  6020.484267: irq_handler_exit:
irq=30 ret=handled
irq/30-can0-346   [000] d......  6020.484493: softirq_raise: vec=3
[action=NET_RX]
irq/30-can0-346   [000] .......  6020.485282: softirq_exit: vec=1
[action=TIMER]
irq/30-can0-346   [000] .......  6020.485306: softirq_entry: vec=3
[action=NET_RX]
irq/30-can0-346   [000] d..h...  6020.485607: irq_handler_entry:
irq=16 name=tc_clkevt
irq/30-can0-346   [000] d..h...  6020.485673: irq_handler_entry:
irq=17 name=at91_rtc
irq/30-can0-346   [000] d..h...  6020.485685: irq_handler_exit:
irq=17 ret=handled
irq/30-can0-346   [000] d..h...  6020.485706: irq_handler_entry:
irq=17 name=atmel_usart_serial.0.auto
irq/30-can0-346   [000] d..h...  6020.485713: irq_handler_exit:
irq=17 ret=handled
irq/30-can0-346   [000] d..h...  6020.485754: irq_handler_exit:
irq=16 ret=handled
irq/30-can0-346   [000] d..h...  6020.486008: irq_handler_entry:
irq=29 name=eth0
irq/30-can0-346   [000] d..h...  6020.486027: irq_handler_exit:
irq=29 ret=handled
irq/30-can0-346   [000] .......  6020.486571: softirq_exit: vec=3
[action=NET_RX]
irq/30-can0-346   [000] d......  6020.486626: at91_irq:
(at91_irq+0x0/0xab0)
irq/30-can0-346   [000] d......  6020.486788: at91_rx_overflow_err:
(at91_irq+0x200/0xab0)