Re: rcu_preempt detected stalls on CPUs/tasks

[Sumit Garg via OP-TEE <op-tee@lists.trustedfirmware.org>]

On Wed, Aug 20, 2025 at 01:10:07PM +0200, Jens Wiklander wrote:
> Hi Thomas,
> 
> On Mon, Aug 4, 2025 at 7:09 PM Stauffer Thomas MTANA via OP-TEE
> <op-tee@lists.trustedfirmware.org> wrote:
> >
> > Hi,
> >
> > I'm running OP-TEE 4.5 with PKCS11TA and ATF lts-v2.12.4 on an iMX8MP. When I create new rsa 4096 bit keypair with OP-TEE, I often get
> >
> > rcu_preempt detected stalls on CPUs/tasks
> >
> > from Linux 6.6.90 (mainline)
> >
> > Also PID 0 is sometimes blocked for more than 30 seconds. When I create a RT task with even higher priority, this process is also blocked up to 2 seconds. For a test I disabled saving/restoring the NS timer register in ATF (arm-trusted-firmware/lib/el3_runtime/aarch64/context_mgmt.c), this seems to get completely rid of the problem. Neither creating nor signing leads to any issue anymore. This hack may lead to other problems I do not fully understand yet. I "believe" that at least since ARMv8, the CPU have their own timers for secure/non-secure world, but I would assume that ATF implements this correctly already.
> 
> I'm starting to suspect that we're setting NS_TIMER_SWITCH to 1 in
> services/spd/opteed/opteed.mk based on a misunderstanding. OP-TEE can
> use timers, but then it's using the EL3 physical timer. So OP-TEE
> should stay off the EL1 physical timer. Sumit, what's your view?

I had to research the history why I added it in the first place. It was
basically added to save and restore cntkctl_el1 register which is needed
for ftrace to work correctly. Have a look here [1]. So your current
proposed patch will break ftrace.

However, as a side effect all the EL1 physical timer registers got saved
and restored which is a problem as mentioned here. So the correct fix
here would be to make NS_TIMER_SWITCH more granular to separate out the
cntkctl_el1 register save and restore.

[1] https://github.com/OP-TEE/optee_os/commit/edaf8c38f534497a65a460f0348a81d3e26b3518

-Sumit

> 
> >
> > Maybe I'm completely wrong here (assuming that it cannot be I'm the first person having this issue on this platform). Hint in any direction would be helpful.
> 
> I'm surprised we haven't seen more of this issue.
> 
> Cheers,
> Jens
> 
> >
> > Regards
> >
> > Thomas
> >