Re: high prioritized threaded interrupt is delayed due to low priority softirq job

["Ahmed S. Darwish" <darwi@linutronix.de>]

On Thu, Nov 17, 2022, yosi yarchi wrote:
>
> 1. SAM9X35 (8 CAN mailboxes),
>

This is a *very* very low-end core (ARM926), so be careful that some
latency issues would be expected.

If you don't have a specific Linux use-case, maybe you can try something
like ZephyrOS instead?

Given that low-end core, and if you're stuck with Linux, a mainline
kernel with preemption disabled (CONFIG_PREEMPT_NONE) might actually be
more beneficial latency-wise than PREEMPT_RT...

> 2. Linux mscb 5.4.41-linux4sam-2020.04-rt24 #1 PREEMPT_RT Wed Nov 9
> 06:12:28 UTC 2022 armv5tejl GNU/Linux

"5.4.41-linux4sam-2020.04-rt24". This is a vendored RT kernel, and
vendored RT kernels are known to have latency issues.

Can you please try official RT kernels instead? More info is here:

  https://wiki.linuxfoundation.org/realtime/preempt_rt_versions

Make sure that debugging options are disabled, e.g.
CONFIG_PROVE_LOCKING, and so on. Afterwards, you'd like to measure the
latencies of your (single-core) system with something like:

  cyclictest                    \
        --default-system        \
        --secaligned            \
        --mlockall              \
        --interval=500          \
        --priority=98           \
        --histogram=2000

And then check the reported worst-case latencies.

> 3. CAN threaded interrupt (irq/30-can0-346) is configured to be
> highest priority on system.

Please not that in RT, many system-critical kernel threads run at rtprio
99. The hight (sane) priority to be assigned should thus be 98.

>
> my problem is that from time to time I get CAN RX overflow.
>

That would be kinda expected with such a low-end core, even when
everything else is "perfect".

>
> I've tried some solutions, no one worked out of the box:
>
> 1. disable all relevant irqs (timer, network) at CAN irq_handler
> (irq_handler_entry: irq=30 name=can0), and enable them at the end of CAN
> threaded interrupt job processing (at91_irq_r)
>

That's wrong in so many ways, please don't do that.

> 2. configure CAN job (at91_irq) to run inside the CAN irq_handler
> (irq_handler_entry: irq=30 name=can0) by adding IRQF_NO_THREAD flag to
> request_irq.

ditto.

Good luck,

--
Ahmed S. Darwish
Linutronix GmbH