Re: [PATCH v4 1/1] powerpc: enable dynamic preemption

[Shrikanth Hegde <sshegde@linux.ibm.com>]

Hi Jirka,
Thanks for these experiments.

On 7/28/26 5:56 AM, Jirka Hladky wrote:
> On Mon, Jul 27, 2026 at 7:10 PM Shrikanth Hegde <sshegde@linux.ibm.com> wrote:
>> That's full preemption mode.
>>
>> When preemption mode changes preempt_enable/disable which were just a
>> barrier earlier now become real preemption points. If the code path
>> repeatedly does the exact same thing, it might pop up.
>> But, can we say is that number expected? it is difficult to put a
>> number to it.
> 
> You were right -- my previous test conflated two variables. I've now
> built a third kernel to separate them. All three are from the same
> 6.15-rc6 source, same machine (POWER10 lp11), no PREEMPT_DYNAMIC:
> 
> Config                                    Mode        PREEMPT_RCU
> kill bogo-ops/sec
> ------------------------------------   ----------  -----------
> -----------------
> PREEMPT_VOLUNTARY=y   voluntary   no            105,014
> PREEMPT_LAZY=y                lazy           no              86,878
> PREEMPT=y                           full             yes             73,317
> 
> voluntary -> lazy                                  -17.3%
> lazy -> full+PREEMPT_RCU               -15.6%
> voluntary -> full+PREEMPT_RCU       -30.2%
> 
> The regression splits roughly 55/45 between the preemption mode
> change and PREEMPT_RCU:
> 
> 1) voluntary -> lazy (-17.3%): preempt_disable/enable becoming real
>     preemption points, as you predicted. On ppc64le this is expensive
>     because the kill() syscall path is very tight and hits these
>     points heavily.
That's true for all archs. Please check your preemption mode in your x86 experiment.
If it remained same, then that small difference could PREEMPT_RCU cost.

even preempt_enable/disable has barriers.

> 
> 2) lazy -> full+PREEMPT_RCU (-15.6%): __rcu_read_lock/__rcu_read_unlock
>     requiring lwsync/isync barriers on ppc64le.
> 

Full is more aggressive in setting in the need_resched bit and
PREEMPT_RCU doing more than just barriers specially __rcu_read_unlock.

>> What I was asking is below. (You can do this only with below 7.0)
>>
>> Config A: CONFIG_PREEMPT_VOLUNTARY=y (CONFIG_PREEMPT_DYNAMIC=n)
>> Config B: CONFIG_PREEMPT_VOLUNTARY=y (CONFIG_PREEMPT_DYNAMIC=y)
> 
> I couldn't do this exact test because 6.15-rc6 doesn't have
> HAVE_PREEMPT_DYNAMIC_KEY for powerpc (that's your 6.16 patch), so
> CONFIG_PREEMPT_DYNAMIC=y would be silently ignored. I would need a
> 6.16-6.19 kernel for this, which is before 7dadeaa6e851 removed
> voluntary as an option.
> 
> But the PREEMPT_LAZY test above achieves the same goal: it isolates
> the preemption mode cost without PREEMPT_RCU.
> 

No. It doesn't confirm. I would recommend you do that case to find out
the cost of PREEMPT_RCU alone.

The reason being, lazy is a real preemption mode. All the callsites of preempt_enable
could force a context switch.

Whereas,
>> Config A: CONFIG_PREEMPT_VOLUNTARY=y (CONFIG_PREEMPT_DYNAMIC=n)
>> Config B: CONFIG_PREEMPT_VOLUNTARY=y (CONFIG_PREEMPT_DYNAMIC=y)

Both are expected to same/similar w.r.t preempt_enable. But as we have discovered
PREEMPT_DYNAMIC in additions enables PREEMPT_RCU. If we get the above data,
then we can quantify the cost due to PREEMPT_RCU alone.


Preemption mode cost is one thing, i.e preempt enable/disable cost,
additional cost is call to schedule itself if need resched is set, that
will likely over weigh the preemption cost.
That is observed in your experiments too.
full is more aggressive in setting the need_resched bit. Plus additional cost of
PREEMPT_RCU which too can set need_resched bits.

>> Are you saying you see regression with voluntary with
>> CONFIG_PREEMPT_DYNAMIC=y?
> 
> Yes. On the ELN kernels with CONFIG_PREEMPT_DYNAMIC=y, the regression
> appears at 6.16 when HAVE_PREEMPT_DYNAMIC_KEY is added, because:
> 1) PREEMPT_DYNAMIC forces the runtime mode to lazy/full (voluntary
>     is no longer available on architectures with ARCH_HAS_PREEMPT_LAZY)

1. Is not true. PREEMPT_DYNAMIC doesn't force the preemption mode switch. You can still choose
voluntary even with PREEMPT_DYNAMIC on 6.16 to 6.19 kernel.

Forced mode switch to lazy/full due to ARCH_HAS_PREEMPT_LAZY has happened in 7.0.


> 2) PREEMPT_DYNAMIC pulls in PREEMPT_RCU
>

This looks it can set need_resched bit aggressive to force a quiescent state?
Concerns i see with this config, I have put it in other thread.

> Both contribute to the ~30% total regression.
> 
> Jirka
>