Re: Preemp-RT kernel builds from 6.3 on fails on Xilinx Zynq 7Z010

[Sebastian Andrzej Siewior <bigeasy@linutronix.de>]

On 2023-05-16 19:13:26 [+0200], Ard Biesheuvel wrote:
> On Tue, 16 May 2023 at 18:33, Sebastian Andrzej Siewior
> <bigeasy@linutronix.de> wrote:
> >
> > + ard, peterz
> 
> Hello Sebastian,
Hi Ard,

> Please check whether 6.4-rc2 is equally affected - we fixed some
> issues related to BH en/disabling from asm code.

oh oh Ard, you are the best. Thank you.

…
> Please take a look, and confirm whether or not we still need to drop
> the get_cpu() call from vfp_sync_hwstate()

Yes, it needs to be dealt with. At the time vfp_sync_hwstate() the
context is fully preemptible. So that get_cpu() needs to be removed
before local_bh_disable(). But…

If I understand the logic right, then the VFP state is saved on context
switch and the FPU access is disabled but the content remains and is in
sync with vfp_current_hw_state. Upon first usage (after the context
switch) in userland there is an exception at which point the access to
the FPU is enabled (FPEXC_EX) and the FPU content is loaded unless it is
in sync as per vfp_current_hw_state.

That means it relies on disabled preemption while operating on
vfp_current_hw_state. On PREEMPT_RT softirqs are only handled in process
context (for instance at the end of the threaded interrupt). Therefore
it is sufficient to disable preemption instead of BH. It would need
something like x86's fpregs_lock().
Otherwise vfp_entry() can get preempted after decisions based on
vfp_current_hw_state have been made. Also kernel_neon_begin() could get
preempted after enabling FPU. I think based on current logic, after
kernel_neon_begin() a task can get preempted on PREEMPT_RT and since
vfp_current_hw_state is NULL the registers won't be saved and a zero set
of registers will be loaded once the neon task gets back on the CPU (it
seems that VFP exceptions in kernel mode are treated the same way as
those in user mode).

What do you think?

> Thanks,
> Ard.

Sebastian