[PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Maulik Shah]

CPU can get IPI interrupt from another CPU while it is executing
cpuidle_select() or about to execute same. The selection do not account
for pending interrupts and may continue to enter selected idle state only
to exit immediately.

Example trace collected when there is cross CPU IPI.

 [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
 [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
 [007] 154.892162: cpu_idle: state=2 cpu_id=7
 [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
 [007] 154.892211: irq_handler_entry: irq=2 name=IPI
 [007] 154.892211: ipi_entry: (Function call interrupts)
 [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
 [007] 154.892214: ipi_exit: (Function call interrupts)

This impacts performance and the above count increments.

commit ccde6525183c ("smp: Introduce a helper function to check for pending
IPIs") already introduced a helper function to check the pending IPIs and
it is used in pmdomain governor to deny the cluster level idle state when
there is a pending IPI on any of cluster CPUs.

This however does not stop CPU to enter CPU level idle state. Make use of
same at CPUidle to deny the idle entry when there is already IPI pending.

With change observing glmark2 [1] off screen scores improving in the range
of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
clusters each with 4 CPUs.

[1] https://github.com/glmark2/glmark2

Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
---
 drivers/cpuidle/cpuidle.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
--- a/drivers/cpuidle/cpuidle.c
+++ b/drivers/cpuidle/cpuidle.c
@@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
 	bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
 	ktime_t time_start, time_end;
 
+	if (cpus_peek_for_pending_ipi(drv->cpumask))
+		return -EBUSY;
+
 	instrumentation_begin();
 
 	/*

---
base-commit: b84a0ebe421ca56995ff78b66307667b62b3a900
change-id: 20260316-cpuidle_ipi-4c64036f9a48

Best regards,
-- 
Maulik Shah <maulik.shah@oss.qualcomm.com>

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Christian Loehle]

On 3/16/26 07:37, Maulik Shah wrote:
> CPU can get IPI interrupt from another CPU while it is executing
> cpuidle_select() or about to execute same. The selection do not account
> for pending interrupts and may continue to enter selected idle state only
> to exit immediately.
> 
> Example trace collected when there is cross CPU IPI.
> 
>  [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>  [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
>  [007] 154.892162: cpu_idle: state=2 cpu_id=7
>  [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
>  [007] 154.892211: irq_handler_entry: irq=2 name=IPI
>  [007] 154.892211: ipi_entry: (Function call interrupts)
>  [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>  [007] 154.892214: ipi_exit: (Function call interrupts)
> 
> This impacts performance and the above count increments.
> 
> commit ccde6525183c ("smp: Introduce a helper function to check for pending
> IPIs") already introduced a helper function to check the pending IPIs and
> it is used in pmdomain governor to deny the cluster level idle state when
> there is a pending IPI on any of cluster CPUs.
> 
> This however does not stop CPU to enter CPU level idle state. Make use of
> same at CPUidle to deny the idle entry when there is already IPI pending.
> 
> With change observing glmark2 [1] off screen scores improving in the range
> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
> clusters each with 4 CPUs.
> 
> [1] https://github.com/glmark2/glmark2
> 
> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
> ---
>  drivers/cpuidle/cpuidle.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
> --- a/drivers/cpuidle/cpuidle.c
> +++ b/drivers/cpuidle/cpuidle.c
> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>  	bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>  	ktime_t time_start, time_end;
>  
> +	if (cpus_peek_for_pending_ipi(drv->cpumask))
> +		return -EBUSY;
> +
>  	instrumentation_begin();
>  
>  	/*
> 
> ---
> base-commit: b84a0ebe421ca56995ff78b66307667b62b3a900
> change-id: 20260316-cpuidle_ipi-4c64036f9a48
> 
> Best regards,

So we already do a per-CPU IPI need_resched() check in the idle path.
Your patch uses drv->cpumask, which will contain all CPUs, preventing idle entry if
any CPU has an IPI pending?

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Maulik Shah (mkshah)]

On 3/16/2026 2:25 PM, Christian Loehle wrote:
> On 3/16/26 07:37, Maulik Shah wrote:
..
..
>>
>> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
>> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
>> --- a/drivers/cpuidle/cpuidle.c
>> +++ b/drivers/cpuidle/cpuidle.c
>> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>>  	bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>>  	ktime_t time_start, time_end;
>>  
>> +	if (cpus_peek_for_pending_ipi(drv->cpumask))
>> +		return -EBUSY;
>> +
>>  	instrumentation_begin();
..
..
>> Best regards,
> 
> So we already do a per-CPU IPI need_resched() check in the idle path.
> Your patch uses drv->cpumask, which will contain all CPUs, preventing idle entry if
> any CPU has an IPI pending?

The IPI interrupt became pending after the need_resched() check,
when the CPUidle governor is selecting a mode, but before the idle entry.

On qualcomm lemans-evk case, drv->cpumask using cpuidle-psci is having only single
CPU but seems other idle drivers may contain all CPUs.

Since intent here was to check the pending IPI on same CPU, this check be replaced
with another need_resched() or using cpus_peek_for_pending_ipi() taking a single
(current) CPU's mask in the argument. I can update this in v2 based on recommendation.

Thanks,
Maulik

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Daniel Lezcano]

On 3/16/26 09:55, Christian Loehle wrote:
> On 3/16/26 07:37, Maulik Shah wrote:
>> CPU can get IPI interrupt from another CPU while it is executing
>> cpuidle_select() or about to execute same. The selection do not account
>> for pending interrupts and may continue to enter selected idle state only
>> to exit immediately.
>>
>> Example trace collected when there is cross CPU IPI.
>>
>>   [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>   [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
>>   [007] 154.892162: cpu_idle: state=2 cpu_id=7
>>   [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
>>   [007] 154.892211: irq_handler_entry: irq=2 name=IPI
>>   [007] 154.892211: ipi_entry: (Function call interrupts)
>>   [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>   [007] 154.892214: ipi_exit: (Function call interrupts)
>>
>> This impacts performance and the above count increments.
>>
>> commit ccde6525183c ("smp: Introduce a helper function to check for pending
>> IPIs") already introduced a helper function to check the pending IPIs and
>> it is used in pmdomain governor to deny the cluster level idle state when
>> there is a pending IPI on any of cluster CPUs.
>>
>> This however does not stop CPU to enter CPU level idle state. Make use of
>> same at CPUidle to deny the idle entry when there is already IPI pending.
>>
>> With change observing glmark2 [1] off screen scores improving in the range
>> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
>> clusters each with 4 CPUs.
>>
>> [1] https://github.com/glmark2/glmark2
>>
>> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
>> ---
>>   drivers/cpuidle/cpuidle.c | 3 +++
>>   1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
>> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
>> --- a/drivers/cpuidle/cpuidle.c
>> +++ b/drivers/cpuidle/cpuidle.c
>> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>>   	bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>>   	ktime_t time_start, time_end;
>>   
>> +	if (cpus_peek_for_pending_ipi(drv->cpumask))
>> +		return -EBUSY;
>> +
>>   	instrumentation_begin();
>>   
>>   	/*
>>
>> ---
>> base-commit: b84a0ebe421ca56995ff78b66307667b62b3a900
>> change-id: 20260316-cpuidle_ipi-4c64036f9a48
>>
>> Best regards,
> 
> So we already do a per-CPU IPI need_resched() check in the idle path.

The need_resched() is not the same check. Here the interrupts are off, 
the test check if there is a pending IPI before entering the sleep 
routine which will in any case abort because of it. This check saves the 
costs related to preparing entering the idle state, the call to the 
firmware and the rollback. Those add an overhead in terms of latency and 
energy for nothing. As stated in the description, this ultimate check 
before going idle was introduced also for the cluster idle state and 
showed a significant improvement [1].

[1] 
https://lore.kernel.org/all/20251105095415.17269-1-ulf.hansson@linaro.org/

> Your patch uses drv->cpumask, which will contain all CPUs, preventing idle entry if
> any CPU has an IPI pending?

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Christian Loehle]

On 3/16/26 09:32, Daniel Lezcano wrote:
> On 3/16/26 09:55, Christian Loehle wrote:
>> On 3/16/26 07:37, Maulik Shah wrote:
>>> CPU can get IPI interrupt from another CPU while it is executing
>>> cpuidle_select() or about to execute same. The selection do not account
>>> for pending interrupts and may continue to enter selected idle state only
>>> to exit immediately.
>>>
>>> Example trace collected when there is cross CPU IPI.
>>>
>>>   [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>>   [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
>>>   [007] 154.892162: cpu_idle: state=2 cpu_id=7
>>>   [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
>>>   [007] 154.892211: irq_handler_entry: irq=2 name=IPI
>>>   [007] 154.892211: ipi_entry: (Function call interrupts)
>>>   [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>>   [007] 154.892214: ipi_exit: (Function call interrupts)
>>>
>>> This impacts performance and the above count increments.
>>>
>>> commit ccde6525183c ("smp: Introduce a helper function to check for pending
>>> IPIs") already introduced a helper function to check the pending IPIs and
>>> it is used in pmdomain governor to deny the cluster level idle state when
>>> there is a pending IPI on any of cluster CPUs.
>>>
>>> This however does not stop CPU to enter CPU level idle state. Make use of
>>> same at CPUidle to deny the idle entry when there is already IPI pending.
>>>
>>> With change observing glmark2 [1] off screen scores improving in the range
>>> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
>>> clusters each with 4 CPUs.
>>>
>>> [1] https://github.com/glmark2/glmark2
>>>
>>> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
>>> ---
>>>   drivers/cpuidle/cpuidle.c | 3 +++
>>>   1 file changed, 3 insertions(+)
>>>
>>> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
>>> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
>>> --- a/drivers/cpuidle/cpuidle.c
>>> +++ b/drivers/cpuidle/cpuidle.c
>>> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>>>       bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>>>       ktime_t time_start, time_end;
>>>   +    if (cpus_peek_for_pending_ipi(drv->cpumask))
>>> +        return -EBUSY;
>>> +
>>>       instrumentation_begin();
>>>         /*
>>>
>>> ---
>>> base-commit: b84a0ebe421ca56995ff78b66307667b62b3a900
>>> change-id: 20260316-cpuidle_ipi-4c64036f9a48
>>>
>>> Best regards,
>>
>> So we already do a per-CPU IPI need_resched() check in the idle path.
> 
> The need_resched() is not the same check. Here the interrupts are off, the test check if there is a pending IPI before entering the sleep routine which will in any case abort because of it. This check saves the costs related to preparing entering the idle state, the call to the firmware and the rollback. Those add an overhead in terms of latency and energy for nothing. As stated in the description, this ultimate check before going idle was introduced also for the cluster idle state and showed a significant improvement [1].
> 
> [1] https://lore.kernel.org/all/20251105095415.17269-1-ulf.hansson@linaro.org/
So I didn't mean this as "it's unnecessary", but it did make me question how big
the "performance" impact of this really is, in particular for per-CPU idle states (i.e.
at most sleep / powerdown for you?)
But if this is only about cluster states (The original patch wasn't really clear on that?)
then one issue is that the non-pmdomain case (e.g. psci PC-mode) we don't actually know
what a cluster is and therefore which CPUs to check for pending IPIs, right?

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Daniel Lezcano]

On 3/16/26 10:50, Christian Loehle wrote:

[ ... ]

>>> So we already do a per-CPU IPI need_resched() check in the idle
>>> path.
>> 
>> The need_resched() is not the same check. Here the interrupts are
>> off, the test check if there is a pending IPI before entering the
>> sleep routine which will in any case abort because of it. This
>> check saves the costs related to preparing entering the idle
>> state, the call to the firmware and the rollback. Those add an
>> overhead in terms of latency and energy for nothing. As stated in
>> the description, this ultimate check before going idle was
>> introduced also for the cluster idle state and showed a
>> significant improvement [1].
>> 
>> [1] https://lore.kernel.org/all/20251105095415.17269-1-
>> ulf.hansson@linaro.org/


> So I didn't mean this as "it's unnecessary", but it did make me
> question how big the "performance" impact of this really is, in
> particular for per-CPU idle states (i.e. at most sleep / powerdown
> for you?)

 From a per CPU perspective, it is hard to say. It really depends on the 
workload, the number of CPUs and the correctness of the governor prediction.

I would say the higher the number of CPUs, the higher the probability to 
receive an IPI, the lesser the accuracy of the governor [1] and the more 
visible the improvement of this change is.

Maulik did some benchmarking with glmark2 and showed an improvement.


> But if this is only about cluster states (The original
> patch wasn't really clear on that?) then one issue is that the non-
> pmdomain case (e.g. psci PC-mode) we don't actually know what a
> cluster is and therefore which CPUs to check for pending IPIs,
> right?

Ulf changes is for platforms, usually Snapdragon, where the kernel has a 
knowledge of the topology and uses the PSCI-OSI (IIRC). So the kernel is 
in charge of the last-man-standing for the group of CPUs belonging to 
the same cluster. It has all the needed information for this specific 
configuration.

In the case of the PSCI-PC, the firmware is in charge of the cluster 
idling and AFAICT it does a test for pending IRQ / IPI.

   -- Daniel


[1] IPI prediction is not possible because it fully depends on the 
scheduler, so on the behavior of the tasks.

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Rafael J. Wysocki]

On Mon, Mar 16, 2026 at 8:38 AM Maulik Shah
<maulik.shah@oss.qualcomm.com> wrote:
>
> CPU can get IPI interrupt from another CPU while it is executing
> cpuidle_select() or about to execute same. The selection do not account
> for pending interrupts and may continue to enter selected idle state only
> to exit immediately.
>
> Example trace collected when there is cross CPU IPI.
>
>  [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>  [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
>  [007] 154.892162: cpu_idle: state=2 cpu_id=7
>  [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
>  [007] 154.892211: irq_handler_entry: irq=2 name=IPI
>  [007] 154.892211: ipi_entry: (Function call interrupts)
>  [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>  [007] 154.892214: ipi_exit: (Function call interrupts)
>
> This impacts performance and the above count increments.
>
> commit ccde6525183c ("smp: Introduce a helper function to check for pending
> IPIs") already introduced a helper function to check the pending IPIs and
> it is used in pmdomain governor to deny the cluster level idle state when
> there is a pending IPI on any of cluster CPUs.

You seem to be overlooking the fact that resched wakeups need not be
signaled via IPIs, but they may be updates of a monitored cache line.

> This however does not stop CPU to enter CPU level idle state. Make use of
> same at CPUidle to deny the idle entry when there is already IPI pending.
>
> With change observing glmark2 [1] off screen scores improving in the range
> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
> clusters each with 4 CPUs.
>
> [1] https://github.com/glmark2/glmark2
>
> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
> ---
>  drivers/cpuidle/cpuidle.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
> --- a/drivers/cpuidle/cpuidle.c
> +++ b/drivers/cpuidle/cpuidle.c
> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>         bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>         ktime_t time_start, time_end;
>
> +       if (cpus_peek_for_pending_ipi(drv->cpumask))
> +               return -EBUSY;
> +

So what if the driver handles all CPUs in the system and there are
many of them (say ~500) and if IPIs occur rarely (because resched
events are not IPIs)?

>         instrumentation_begin();
>
>         /*
>
> ---

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Maulik Shah (mkshah)]

On 3/20/2026 11:59 PM, Rafael J. Wysocki wrote:
> On Mon, Mar 16, 2026 at 8:38 AM Maulik Shah
> <maulik.shah@oss.qualcomm.com> wrote:
>>
>> CPU can get IPI interrupt from another CPU while it is executing
>> cpuidle_select() or about to execute same. The selection do not account
>> for pending interrupts and may continue to enter selected idle state only
>> to exit immediately.
>>
>> Example trace collected when there is cross CPU IPI.
>>
>>  [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>  [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
>>  [007] 154.892162: cpu_idle: state=2 cpu_id=7
>>  [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
>>  [007] 154.892211: irq_handler_entry: irq=2 name=IPI
>>  [007] 154.892211: ipi_entry: (Function call interrupts)
>>  [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>  [007] 154.892214: ipi_exit: (Function call interrupts)
>>
>> This impacts performance and the above count increments.
>>
>> commit ccde6525183c ("smp: Introduce a helper function to check for pending
>> IPIs") already introduced a helper function to check the pending IPIs and
>> it is used in pmdomain governor to deny the cluster level idle state when
>> there is a pending IPI on any of cluster CPUs.
> 
> You seem to be overlooking the fact that resched wakeups need not be
> signaled via IPIs, but they may be updates of a monitored cache line.
> 
>> This however does not stop CPU to enter CPU level idle state. Make use of
>> same at CPUidle to deny the idle entry when there is already IPI pending.
>>
>> With change observing glmark2 [1] off screen scores improving in the range
>> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
>> clusters each with 4 CPUs.
>>
>> [1] https://github.com/glmark2/glmark2
>>
>> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
>> ---
>>  drivers/cpuidle/cpuidle.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
>> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
>> --- a/drivers/cpuidle/cpuidle.c
>> +++ b/drivers/cpuidle/cpuidle.c
>> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>>         bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>>         ktime_t time_start, time_end;
>>
>> +       if (cpus_peek_for_pending_ipi(drv->cpumask))
>> +               return -EBUSY;
>> +
> 
> So what if the driver handles all CPUs in the system and there are
> many of them (say ~500) and if IPIs occur rarely (because resched
> events are not IPIs)?

Missed the case of driver handling multiple CPUs,
In v2 would fix this as below, which checks pending IPI on single
CPU trying to enter idle.

     if (cpus_peek_for_pending_ipi(cpumask_of(dev->cpu)))

I see IPIs do occur often, in the glmark2 offscreen case
mentioned in commit text, out of total ~12.2k IPIs across all 8 CPUs,
~9.6k are function call IPIs, ~2k are IRQ work IPIs, ~560 Timer broadcast
IPIs while rescheduling IPIs are only 82.

Thanks,
Maulik

> 
>>         instrumentation_begin();
>>
>>         /*
>>
>> ---

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Ulf Hansson]

On Mon, 16 Mar 2026 at 08:38, Maulik Shah <maulik.shah@oss.qualcomm.com> wrote:
>
> CPU can get IPI interrupt from another CPU while it is executing
> cpuidle_select() or about to execute same. The selection do not account
> for pending interrupts and may continue to enter selected idle state only
> to exit immediately.
>
> Example trace collected when there is cross CPU IPI.
>
>  [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>  [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
>  [007] 154.892162: cpu_idle: state=2 cpu_id=7
>  [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
>  [007] 154.892211: irq_handler_entry: irq=2 name=IPI
>  [007] 154.892211: ipi_entry: (Function call interrupts)
>  [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>  [007] 154.892214: ipi_exit: (Function call interrupts)
>
> This impacts performance and the above count increments.
>
> commit ccde6525183c ("smp: Introduce a helper function to check for pending
> IPIs") already introduced a helper function to check the pending IPIs and
> it is used in pmdomain governor to deny the cluster level idle state when
> there is a pending IPI on any of cluster CPUs.
>
> This however does not stop CPU to enter CPU level idle state. Make use of
> same at CPUidle to deny the idle entry when there is already IPI pending.
>
> With change observing glmark2 [1] off screen scores improving in the range
> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
> clusters each with 4 CPUs.
>
> [1] https://github.com/glmark2/glmark2
>
> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
> ---
>  drivers/cpuidle/cpuidle.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
> --- a/drivers/cpuidle/cpuidle.c
> +++ b/drivers/cpuidle/cpuidle.c
> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>         bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>         ktime_t time_start, time_end;
>
> +       if (cpus_peek_for_pending_ipi(drv->cpumask))
> +               return -EBUSY;

As other reviews already pointed out, this must be called only for the
current CPU.

That said, did you play with bailing out just before the call to the
target_state->enter()? It would be interesting to know if that changes
the "stats" somehow.

> +
>         instrumentation_begin();
>
>         /*
>
> ---
> base-commit: b84a0ebe421ca56995ff78b66307667b62b3a900
> change-id: 20260316-cpuidle_ipi-4c64036f9a48
>
> Best regards,
> --
> Maulik Shah <maulik.shah@oss.qualcomm.com>
>

Kind regards
Uffe

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Rafael J. Wysocki]

On Mon, Mar 23, 2026 at 1:13 PM Maulik Shah (mkshah)
<maulik.shah@oss.qualcomm.com> wrote:
>
>
>
> On 3/20/2026 11:59 PM, Rafael J. Wysocki wrote:
> > On Mon, Mar 16, 2026 at 8:38 AM Maulik Shah
> > <maulik.shah@oss.qualcomm.com> wrote:
> >>
> >> CPU can get IPI interrupt from another CPU while it is executing
> >> cpuidle_select() or about to execute same. The selection do not account
> >> for pending interrupts and may continue to enter selected idle state only
> >> to exit immediately.
> >>
> >> Example trace collected when there is cross CPU IPI.
> >>
> >>  [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
> >>  [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
> >>  [007] 154.892162: cpu_idle: state=2 cpu_id=7
> >>  [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
> >>  [007] 154.892211: irq_handler_entry: irq=2 name=IPI
> >>  [007] 154.892211: ipi_entry: (Function call interrupts)
> >>  [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
> >>  [007] 154.892214: ipi_exit: (Function call interrupts)
> >>
> >> This impacts performance and the above count increments.
> >>
> >> commit ccde6525183c ("smp: Introduce a helper function to check for pending
> >> IPIs") already introduced a helper function to check the pending IPIs and
> >> it is used in pmdomain governor to deny the cluster level idle state when
> >> there is a pending IPI on any of cluster CPUs.
> >
> > You seem to be overlooking the fact that resched wakeups need not be
> > signaled via IPIs, but they may be updates of a monitored cache line.
> >
> >> This however does not stop CPU to enter CPU level idle state. Make use of
> >> same at CPUidle to deny the idle entry when there is already IPI pending.
> >>
> >> With change observing glmark2 [1] off screen scores improving in the range
> >> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
> >> clusters each with 4 CPUs.
> >>
> >> [1] https://github.com/glmark2/glmark2
> >>
> >> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
> >> ---
> >>  drivers/cpuidle/cpuidle.c | 3 +++
> >>  1 file changed, 3 insertions(+)
> >>
> >> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
> >> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
> >> --- a/drivers/cpuidle/cpuidle.c
> >> +++ b/drivers/cpuidle/cpuidle.c
> >> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
> >>         bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
> >>         ktime_t time_start, time_end;
> >>
> >> +       if (cpus_peek_for_pending_ipi(drv->cpumask))
> >> +               return -EBUSY;
> >> +
> >
> > So what if the driver handles all CPUs in the system and there are
> > many of them (say ~500) and if IPIs occur rarely (because resched
> > events are not IPIs)?
>
> Missed the case of driver handling multiple CPUs,
> In v2 would fix this as below, which checks pending IPI on single
> CPU trying to enter idle.
>
>      if (cpus_peek_for_pending_ipi(cpumask_of(dev->cpu)))

And the for_each_cpu() loop in cpus_peek_for_pending_ipi() would then
become useless overhead, wouldn't ir?

> I see IPIs do occur often, in the glmark2 offscreen case
> mentioned in commit text, out of total ~12.2k IPIs across all 8 CPUs,
> ~9.6k are function call IPIs, ~2k are IRQ work IPIs, ~560 Timer broadcast
> IPIs while rescheduling IPIs are only 82.

So how many of those IPIs actually wake up CPUs from idle prematurely?

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Maulik Shah (mkshah)]

On 3/24/2026 9:37 PM, Rafael J. Wysocki wrote:
> On Mon, Mar 23, 2026 at 1:13 PM Maulik Shah (mkshah)
> <maulik.shah@oss.qualcomm.com> wrote:
>>
>>
>>
>> On 3/20/2026 11:59 PM, Rafael J. Wysocki wrote:
>>> On Mon, Mar 16, 2026 at 8:38 AM Maulik Shah
>>> <maulik.shah@oss.qualcomm.com> wrote:
>>>>
>>>> CPU can get IPI interrupt from another CPU while it is executing
>>>> cpuidle_select() or about to execute same. The selection do not account
>>>> for pending interrupts and may continue to enter selected idle state only
>>>> to exit immediately.
>>>>
>>>> Example trace collected when there is cross CPU IPI.
>>>>
>>>>  [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>>>  [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
>>>>  [007] 154.892162: cpu_idle: state=2 cpu_id=7
>>>>  [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
>>>>  [007] 154.892211: irq_handler_entry: irq=2 name=IPI
>>>>  [007] 154.892211: ipi_entry: (Function call interrupts)
>>>>  [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>>>  [007] 154.892214: ipi_exit: (Function call interrupts)
>>>>
>>>> This impacts performance and the above count increments.
>>>>
>>>> commit ccde6525183c ("smp: Introduce a helper function to check for pending
>>>> IPIs") already introduced a helper function to check the pending IPIs and
>>>> it is used in pmdomain governor to deny the cluster level idle state when
>>>> there is a pending IPI on any of cluster CPUs.
>>>
>>> You seem to be overlooking the fact that resched wakeups need not be
>>> signaled via IPIs, but they may be updates of a monitored cache line.
>>>
>>>> This however does not stop CPU to enter CPU level idle state. Make use of
>>>> same at CPUidle to deny the idle entry when there is already IPI pending.
>>>>
>>>> With change observing glmark2 [1] off screen scores improving in the range
>>>> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
>>>> clusters each with 4 CPUs.
>>>>
>>>> [1] https://github.com/glmark2/glmark2
>>>>
>>>> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
>>>> ---
>>>>  drivers/cpuidle/cpuidle.c | 3 +++
>>>>  1 file changed, 3 insertions(+)
>>>>
>>>> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
>>>> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
>>>> --- a/drivers/cpuidle/cpuidle.c
>>>> +++ b/drivers/cpuidle/cpuidle.c
>>>> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>>>>         bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>>>>         ktime_t time_start, time_end;
>>>>
>>>> +       if (cpus_peek_for_pending_ipi(drv->cpumask))
>>>> +               return -EBUSY;
>>>> +
>>>
>>> So what if the driver handles all CPUs in the system and there are
>>> many of them (say ~500) and if IPIs occur rarely (because resched
>>> events are not IPIs)?
>>
>> Missed the case of driver handling multiple CPUs,
>> In v2 would fix this as below, which checks pending IPI on single
>> CPU trying to enter idle.
>>
>>      if (cpus_peek_for_pending_ipi(cpumask_of(dev->cpu)))
> 
> And the for_each_cpu() loop in cpus_peek_for_pending_ipi() would then
> become useless overhead, wouldn't ir?

Given that mask in loop for_each_cpu(cpu, mask) will have a single cpu, overhead should be minor.
or May be we can have new API for a single CPU case.

> 
>> I see IPIs do occur often, in the glmark2 offscreen case
>> mentioned in commit text, out of total ~12.2k IPIs across all 8 CPUs,
>> ~9.6k are function call IPIs, ~2k are IRQ work IPIs, ~560 Timer broadcast
>> IPIs while rescheduling IPIs are only 82.
> 
> So how many of those IPIs actually wake up CPUs from idle prematurely?

282 times out of total 12.2k IPIs (~2.3%) hitting the newly added condition.

Thanks,
Maulik

Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending

[Maulik Shah (mkshah)]

On 3/24/2026 9:16 PM, Ulf Hansson wrote:
> On Mon, 16 Mar 2026 at 08:38, Maulik Shah <maulik.shah@oss.qualcomm.com> wrote:
>>
>> CPU can get IPI interrupt from another CPU while it is executing
>> cpuidle_select() or about to execute same. The selection do not account
>> for pending interrupts and may continue to enter selected idle state only
>> to exit immediately.
>>
>> Example trace collected when there is cross CPU IPI.
>>
>>  [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>  [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts)
>>  [007] 154.892162: cpu_idle: state=2 cpu_id=7
>>  [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7
>>  [007] 154.892211: irq_handler_entry: irq=2 name=IPI
>>  [007] 154.892211: ipi_entry: (Function call interrupts)
>>  [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007
>>  [007] 154.892214: ipi_exit: (Function call interrupts)
>>
>> This impacts performance and the above count increments.
>>
>> commit ccde6525183c ("smp: Introduce a helper function to check for pending
>> IPIs") already introduced a helper function to check the pending IPIs and
>> it is used in pmdomain governor to deny the cluster level idle state when
>> there is a pending IPI on any of cluster CPUs.
>>
>> This however does not stop CPU to enter CPU level idle state. Make use of
>> same at CPUidle to deny the idle entry when there is already IPI pending.
>>
>> With change observing glmark2 [1] off screen scores improving in the range
>> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two
>> clusters each with 4 CPUs.
>>
>> [1] https://github.com/glmark2/glmark2
>>
>> Signed-off-by: Maulik Shah <maulik.shah@oss.qualcomm.com>
>> ---
>>  drivers/cpuidle/cpuidle.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
>> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644
>> --- a/drivers/cpuidle/cpuidle.c
>> +++ b/drivers/cpuidle/cpuidle.c
>> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev,
>>         bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP);
>>         ktime_t time_start, time_end;
>>
>> +       if (cpus_peek_for_pending_ipi(drv->cpumask))
>> +               return -EBUSY;
> 
> As other reviews already pointed out, this must be called only for the
> current CPU.

Yes, addressing in v2.

> 
> That said, did you play with bailing out just before the call to the
> target_state->enter()? It would be interesting to know if that changes
> the "stats" somehow.

Yes, i did play moving this and "stats" do change with differences in next idle state selection.

below is high level scenario happening when the IPI deny change is placed inside target_state->enter()
or any place which will update the rejected count.

Using a menu governor,

 entered_state = target_state->enter(dev, drv, index);

 - entered_state will be negative error when IPI is pending.

 - This makes rejected count increment (which is okay) but it also makes dev->last_residency_ns = 0;

 - For next idle entry, menu governor will log this last interval as failed,
   via menu_select() -> menu_update_intervals(data, UINT_MAX);
   this is because menu_reflect() is not invoked for failed entry.

 - This makes 1 of last 8 intervals invalid, which don't get accounted from get_typical_interval(),
   hence divisor value in this API is never reaching 8, the goto again, loop inside get_typical_interval()
   will get aborted "faster". This interval logging have good influence on next 8 idle entries until
   it will get replaced with meaningful value.

 - In ideal case get_typical_interval() would go from divisor value reaching 8, 7 and then 6 and once its down to 6,
   it gets aborted if not predicted yet, so maximum 3 trials, but with any interval invalid
   it finishes faster, often without prediction.

 - sometimes IPI bailouts may happen frequently, in such cases we have 3 to 4 intervals as invalid in history,
   effectively making get_typical_interval() not predicting since divisor value in first loop would be less than 6.

 - Not predicting via get_typical_interval() can make deeper idle selection more often by only going with
   adjusted sleep lengths.

In summary, 

The benefits seen by aborting the idle entry with IPI pending getting nullified when it is logged
as "rejected" due to next idle entries can go deeper.

If we ignore setting dev->last_residency_ns = 0 for rejected cases or make menu_select() ignore the
last rejected iteration to update in intervals history, this would improve the chances of get_typical_interval()
to predict a meaningful sleep length (in a separate change).

For this change, IMO its better to bail out early without logging, since CPU did not make entry to idle driver yet.

Thanks,
Maulik

> 
>> +
>>         instrumentation_begin();
>>
>>         /*
>>
>> ---
>> base-commit: b84a0ebe421ca56995ff78b66307667b62b3a900
>> change-id: 20260316-cpuidle_ipi-4c64036f9a48
>>
>> Best regards,
>> --
>> Maulik Shah <maulik.shah@oss.qualcomm.com>
>>
> 
> Kind regards
> Uffe