Re: [PATCH v14 09/13] sched/debug: Add migration stats due to non preferred CPUs

Re: [PATCH v14 09/13] sched/debug: Add migration stats due to non preferred CPUs

[Nathan Chancellor]

On Mon, Sep 28, 2026 at 11:07:24AM +0530, Shrikanth Hegde wrote:
> Add a new per-task stat,
> - nr_migrations_cpu_non_preferred: number of push migrations while the
>   CPU is non-preferred.
> 
> Since this new stat is per-task, it changes only /proc/<pid>/sched.
> It doesn't update /proc/schedstat. Hence increasing the schedstat version
> is not necessary.
> 
> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
...
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 5049eff58fb7..7465c983e6f6 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -11236,8 +11236,13 @@ static int sched_non_preferred_cpu_push_stop(void *arg)
>  		update_rq_clock(rq);
>  		context_unsafe_alias(rq);
>  
> -		if (task_rq(p) == rq && task_on_rq_queued(p))
> -			rq = __migrate_task(rq, &rf, p, cpu);
> +		if (task_rq(p) == rq && task_on_rq_queued(p)) {
> +			struct rq *dest_rq = __migrate_task(rq, &rf, p, cpu);
> +
> +			if (rq != dest_rq)
> +				schedstat_inc(p->stats.nr_migrations_cpu_non_preferred);
> +			rq = dest_rq;
> +		}
>  		rq_unlock(rq, &rf);
>  	}

This breaks the build for me with clang-23+ (which have context analysis
enabled by default), although it bisects to the final patch of the
series since this is under CONFIG_PREFERRED_CPU and it is not selected
until then.

  kernel/sched/core.c:11292:25: error: calling function '__migrate_task' requires holding raw_spinlock 'rq_lockp(rq)' exclusively [-Werror,-Wthread-safety-analysis]
   11292 |                         struct rq *dest_rq = __migrate_task(rq, &rf, p, cpu);
         |                                              ^
  kernel/sched/core.c:11298:3: error: releasing raw_spinlock 'rq_lockp(rq)' that was not held [-Werror,-Wthread-safety-analysis]
   11298 |                 rq_unlock(rq, &rf);
         |                 ^
  kernel/sched/core.c:11303:1: error: raw_spinlock 'rq_lockp(__this_rq())' is not held on every path through here [-Werror,-Wthread-safety-analysis]
   11303 | }
         | ^
  kernel/sched/core.c:11286:3: note: raw_spinlock acquired here
   11286 |                 rq_lock(rq, &rf);
         |                 ^
  3 errors generated.

Not sure what the proper fix for this is, maybe another
context_unsafe_alias()? cc Marco just in case

-- 
Cheers,
Nathan

Re: [PATCH v14 09/13] sched/debug: Add migration stats due to non preferred CPUs

[Shrikanth Hegde]

Hi Nathan. Thanks for report.

On 9/29/26 5:48 PM, Nathan Chancellor wrote:
> On Mon, Sep 28, 2026 at 11:07:24AM +0530, Shrikanth Hegde wrote:
>> Add a new per-task stat,
>> - nr_migrations_cpu_non_preferred: number of push migrations while the
>>    CPU is non-preferred.
>>
>> Since this new stat is per-task, it changes only /proc/<pid>/sched.
>> It doesn't update /proc/schedstat. Hence increasing the schedstat version
>> is not necessary.
>>
>> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
> ...
>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> index 5049eff58fb7..7465c983e6f6 100644
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -11236,8 +11236,13 @@ static int sched_non_preferred_cpu_push_stop(void *arg)
>>   		update_rq_clock(rq);
>>   		context_unsafe_alias(rq);
>>   
>> -		if (task_rq(p) == rq && task_on_rq_queued(p))
>> -			rq = __migrate_task(rq, &rf, p, cpu);
>> +		if (task_rq(p) == rq && task_on_rq_queued(p)) {
>> +			struct rq *dest_rq = __migrate_task(rq, &rf, p, cpu);
>> +
>> +			if (rq != dest_rq)
>> +				schedstat_inc(p->stats.nr_migrations_cpu_non_preferred);
>> +			rq = dest_rq;
>> +		}
>>   		rq_unlock(rq, &rf);
>>   	}
> 
> This breaks the build for me with clang-23+ (which have context analysis
> enabled by default), although it bisects to the final patch of the
> series since this is under CONFIG_PREFERRED_CPU and it is not selected
> until then.
> 
>    kernel/sched/core.c:11292:25: error: calling function '__migrate_task' requires holding raw_spinlock 'rq_lockp(rq)' exclusively [-Werror,-Wthread-safety-analysis]
>     11292 |                         struct rq *dest_rq = __migrate_task(rq, &rf, p, cpu);
>           |                                              ^
>    kernel/sched/core.c:11298:3: error: releasing raw_spinlock 'rq_lockp(rq)' that was not held [-Werror,-Wthread-safety-analysis]
>     11298 |                 rq_unlock(rq, &rf);
>           |                 ^
>    kernel/sched/core.c:11303:1: error: raw_spinlock 'rq_lockp(__this_rq())' is not held on every path through here [-Werror,-Wthread-safety-analysis]
>     11303 | }
>           | ^
>    kernel/sched/core.c:11286:3: note: raw_spinlock acquired here
>     11286 |                 rq_lock(rq, &rf);
>           |                 ^
>    3 errors generated.
> 
> Not sure what the proper fix for this is, maybe another
> context_unsafe_alias()? cc Marco just in case
> 

I suspect it is due to using of dest_rq = rq and rq is changing context.

let me try locally and see the fix.
One fix is use rq->cpu instead of rq comparison to see if migration happened.

Re: [PATCH v14 09/13] sched/debug: Add migration stats due to non preferred CPUs

[Shrikanth Hegde]

Hi Nathan.

On 9/29/26 6:13 PM, Shrikanth Hegde wrote:
> Hi Nathan. Thanks for report.
> 
> On 9/29/26 5:48 PM, Nathan Chancellor wrote:
>> On Mon, Sep 28, 2026 at 11:07:24AM +0530, Shrikanth Hegde wrote:
>>> Add a new per-task stat,
>>> - nr_migrations_cpu_non_preferred: number of push migrations while the
>>>    CPU is non-preferred.
>>>
>>> Since this new stat is per-task, it changes only /proc/<pid>/sched.
>>> It doesn't update /proc/schedstat. Hence increasing the schedstat version
>>> is not necessary.
>>>
>>> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
>> ...
>>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>>> index 5049eff58fb7..7465c983e6f6 100644
>>> --- a/kernel/sched/core.c
>>> +++ b/kernel/sched/core.c
>>> @@ -11236,8 +11236,13 @@ static int sched_non_preferred_cpu_push_stop(void *arg)
>>>           update_rq_clock(rq);
>>>           context_unsafe_alias(rq);
>>> -        if (task_rq(p) == rq && task_on_rq_queued(p))
>>> -            rq = __migrate_task(rq, &rf, p, cpu);
>>> +        if (task_rq(p) == rq && task_on_rq_queued(p)) {
>>> +            struct rq *dest_rq = __migrate_task(rq, &rf, p, cpu);
>>> +
>>> +            if (rq != dest_rq)
>>> +                schedstat_inc(p->stats.nr_migrations_cpu_non_preferred);
>>> +            rq = dest_rq;
>>> +        }
>>>           rq_unlock(rq, &rf);
>>>       }
>>
>> This breaks the build for me with clang-23+ (which have context analysis
>> enabled by default), although it bisects to the final patch of the
>> series since this is under CONFIG_PREFERRED_CPU and it is not selected
>> until then.
>>
>>    kernel/sched/core.c:11292:25: error: calling function '__migrate_task' requires holding raw_spinlock 'rq_lockp(rq)' exclusively [-Werror,-Wthread-safety-analysis]
>>     11292 |                         struct rq *dest_rq = __migrate_task(rq, &rf, p, cpu);
>>           |                                              ^
>>    kernel/sched/core.c:11298:3: error: releasing raw_spinlock 'rq_lockp(rq)' that was not held [-Werror,-Wthread-safety-analysis]
>>     11298 |                 rq_unlock(rq, &rf);
>>           |                 ^
>>    kernel/sched/core.c:11303:1: error: raw_spinlock 'rq_lockp(__this_rq())' is not held on every path through here [-Werror,-Wthread-safety-analysis]
>>     11303 | }
>>           | ^
>>    kernel/sched/core.c:11286:3: note: raw_spinlock acquired here
>>     11286 |                 rq_lock(rq, &rf);
>>           |                 ^
>>    3 errors generated.
>>
>> Not sure what the proper fix for this is, maybe another
>> context_unsafe_alias()? cc Marco just in case
>>
> 
> I suspect it is due to using of dest_rq = rq and rq is changing context.
> 
> let me try locally and see the fix.
> One fix is use rq->cpu instead of rq comparison to see if migration happened.

Well, it was rather due to Patch 8/13 which had rq marked as context_unsafe_alias
after acquiring it.

I have tried below and that helps to fix the warnings.
I will write a changelog and send it across soon.

Let me know if it works for you.

---

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 0bb86a43a592..23677d76f9d2 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -11283,10 +11283,10 @@ static int sched_non_preferred_cpu_push_stop(void *arg)
  		 * safely bail out.
  		 */
  		cpu = select_fallback_rq(rq->cpu, p);
+		context_unsafe_alias(rq);
  		rq_lock(rq, &rf);
  		rq->npc_push_work_pending = false;
  		update_rq_clock(rq);
-		context_unsafe_alias(rq);
  
  		if (task_rq(p) == rq && task_on_rq_queued(p)) {
  			struct rq *dest_rq = __migrate_task(rq, &rf, p, cpu);