Re: [PATCH v4 1/1] powerpc: enable dynamic preemption

[Shrikanth Hegde <sshegde@linux.ibm.com>]

Hi Jirka.

On 7/27/26 10:20 PM, Jirka Hladky wrote:
> On Mon, Jul 27, 2026 at 6:16 PM Paul E. McKenney <paulmck@kernel.org> wrote:
>> Yes, non-preemptible RCU's __rcu_read_{,un}lock() are (almost) no-ops,
>> but preemptible RCU must actually execute real code.  But I would not
>> expect *this* much overhead.
> 

Thanks for doing these experiment.

> I've now isolated the PREEMPT_RCU cost with a controlled experiment.
> Built two kernels from the same 6.15-rc6 upstream source on the same
> POWER10 machine, no CONFIG_PREEMPT_DYNAMIC in either case:
> 
> Config A: CONFIG_PREEMPT_VOLUNTARY=y (no PREEMPT_RCU)
> Config B: CONFIG_PREEMPT=y           (PREEMPT_RCU=y)  

That's full preemption mode.

> 
> Results (stress-ng --kill 1 -t 23, SELinux enforcing):
> 
> Kernel                       PREEMPT_RCU   kill bogo-ops/sec
> ---------------------------  -----------   -----------------
> 6.15-rc6-voluntary-test      no            105,014
> 6.15-rc6-preempt-test        yes            73,317
> Delta                                      -30.2%
> 

That is voluntary -> full preemption change.

When preemption mode changes preempt_enable/disable which were just a
barrier earlier now become real preemption points. If the code path
repeatedly does the exact same thing, it might pop up.
But, can we say is that number expected? it is difficult to put a number to it.

What I was asking is below. (You can do this only with below 7.0)

Config A: CONFIG_PREEMPT_VOLUNTARY=y (CONFIG_PREEMPT_DYNAMIC=n i.e no PREEMPT_RCU)
Config B: CONFIG_PREEMPT_VOLUNTARY=y (CONFIG_PREEMPT_DYNAMIC=y i.e PREEMPT_RCU)

That should keep in voluntary preemption.
You can confirm with dynamic preemption using /sys/kerenl/debug/sched/preempt.

In Config B, though there is rcu_read_lock/unlock it should be just a barrier.


> PREEMPT_RCU alone accounts for ~30% on this workload. The kill()
> path hits rcu_read_lock/unlock very heavily through the SELinux AVC
> (avc_has_perm -> avc_lookup wraps every hash table lookup in an RCU
> read-side critical section).
> 
> This also answers Shrikanth's question about whether the regression
> is from the preemption mode change (voluntary -> lazy) or from
> PREEMPT_RCU. Since no PREEMPT_DYNAMIC is involved in either build,
> the preemption mode is not a factor. Additionally, switching between
> full and lazy at runtime on 7.1 showed only ~1% difference (57,476
> vs 56,892), further confirming the mode doesn't matter.

Lazy/full switch is same. There will not be any additional overhead.
The real concern is none/voluntary vs lazy/full.

> 
>> OK, if you are executing an isync or an lwsync instruction in each call
>> to __rcu_read_{,un}lock(), that would explain the overhead.
> 
> Yes, that's what perf shows. __rcu_read_lock and __rcu_read_unlock
> together consume ~9% of total cycles on ppc64le with PREEMPT_RCU,
> vs essentially 0% without.
> 
>> CONFIG_PREEMPT_DYNAMIC=n for the win?
> 
> That's the simplest distro workaround for ppc64le. But since commit
> 7dadeaa6e851 ("sched: Further restrict the preemption modes")
> removed PREEMPT_VOLUNTARY as an option for architectures with
> ARCH_HAS_PREEMPT_LAZY (which includes powerpc), distros that want
> voluntary preemption on ppc64le would need to disable PREEMPT_DYNAMIC
> anyway.
> 
> Is there any path to reducing the barrier cost in
> __rcu_read_lock/__rcu_read_unlock on weakly-ordered architectures?
> Or is the current implementation fundamentally constrained by the
> memory model?
> 

Are you saying you see regression with voluntary with CONFIG_PREEMPT_DYNAMIC=y?

> Jirka
>