Re: Raspberry Pi 5 and PREEMPT_RT (6.13.0-rc3)

[Sebastian Andrzej Siewior <bigeasy@linutronix.de>]

On 2024-12-20 14:18:43 [+0100], Florian Paul Schmidt wrote:
> Hi!
Hi,

> I'm playing around with a Raspberry Pi 5 and a PREEMPT_RT kernel (raspberry
> pi 6.13.y branch) and wonder about the latencies I see. First I tried
> cyclictest and saw latencies in the upper 50s up to 80s or so under a
> kernel compile load.
> 
> So I compiled the kernel with timerlat and osnoise tracers and ran a bit of
> 
> rtla timerlat top -a 60 --dma-latency 0
> 
> and I get traces like the following:
…
> ## CPU 3 hit stop tracing, analyzing it ##
>   IRQ handler delay:                                         0.00 us (0.00 %)
>   IRQ latency:                                               5.16 us
>   Timerlat IRQ duration:                                    47.41 us (42.53 %)
>   Blocking thread:                                          50.50 us (45.30 %)
>                           stress:278226                     50.50 us
>     Blocking thread stack trace
>                 -> timerlat_irq
>                 -> __hrtimer_run_queues
>                 -> hrtimer_interrupt
>                 -> arch_timer_handler_phys
>                 -> handle_percpu_devid_irq
>                 -> generic_handle_domain_irq
>                 -> gic_handle_irq
>                 -> call_on_irq_stack
>                 -> do_interrupt_handler
>                 -> el1_interrupt
>                 -> el1h_64_irq_handler
>                 -> el1h_64_irq
>                 -> folios_put_refs
…
>   Thread latency:                                          111.47 us (100%)

The backtrace appears long but everything up to folios_put_refs()
appears preemtible. Then there is the interrupt handler. This looks
okay.

> Max timerlat IRQ latency from idle: 1.98 us in cpu 3
>   Saving trace to timerlat_trace.txt
> 
> 
> OK, here at least something related to virtual memory is happening.
> I still wonder though: How come the latencies are so large? Is the
> raspberry pi platform just not suited to reach lower latencies? Is
> everything working as expected? Are maybe the raspberry pi
> patches responsible for latency sources which are invisible in the
> traces? I there any hope for getting these latencies a little more
> under control?

Based on this trace, I don't see a section which disables preemption/
interrupts and be the reason for the interrupt happening.

One thing that could delay your doing are small caches.
Then there is looking within the system: An inaccurate/ low resolution
clocksource and/ or timer device.

> Kind regards,
> FPS

Sebastian