Re: [PATCH] sched/topology: clear freecpu bit on detach

[Florian Fainelli <f.fainelli@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1707 bytes --]



On 4/22/2025 9:48 PM, Doug Berger wrote:
> There is a hazard in the deadline scheduler where an offlined CPU
> can have its free_cpus bit left set in the def_root_domain when
> the schedutil cpufreq governor is used. This can allow a deadline
> thread to be pushed to the runqueue of a powered down CPU which
> breaks scheduling. The details can be found here:
> https://lore.kernel.org/lkml/20250110233010.2339521-1-opendmb@gmail.com
> 
> The free_cpus mask is expected to be cleared by set_rq_offline();
> however, the hazard occurs before the root domain is made online
> during CPU hotplug so that function is not invoked for the CPU
> that is being made active.
> 
> This commit works around the issue by ensuring the free_cpus bit
> for a CPU is always cleared when the CPU is removed from a
> root_domain. This likely makes the call of cpudl_clear_freecpu()
> in rq_offline_dl() fully redundant, but I have not removed it
> here because I am not certain of all flows.
> 
> It seems likely that a better solution is possible from someone
> more familiar with the scheduler implementation, but this
> approach is minimally invasive from someone who is not.
> 
> Signed-off-by: Doug Berger <opendmb@gmail.com>
> ---

FWIW, we were able to reproduce this with the attached hotplug.sh script 
which would just randomly hot plug/unplug CPUs (./hotplug.sh 4). Within 
a few hundred of iterations you could see the lock up occur, it's 
unclear why this has not been seen by more people.

Since this is not the first posting or attempt at fixing this bug [1] 
and we consider it to be a serious one, can this be reviewed/commented 
on/applied? Thanks!

[1]: https://lkml.org/lkml/2025/1/14/1687
-- 
Florian

[-- Attachment #2: hotplug.sh --]
[-- Type: text/plain, Size: 699 bytes --]

#!/bin/sh
# Hotplug test

usage() {
	echo "Usage: $0 [# cpus]"
	echo "   If number of cpus is not given, defaults to 2"
	exit
}

# Default to 2 CPUs
NR_CPUS=${1:-2}

[ $NR_CPUS -lt 2 ] && usage 1>&2

MAXCPU=$((NR_CPUS-1))
MAX=`cat /sys/devices/system/cpu/kernel_max`

[ $MAXCPU -gt $MAX ] && echo "Too many CPUs" 1>&2 && usage 1>&2

cpu_path() {
	echo /sys/devices/system/cpu/cpu$1
}

checkpoint_test() {
	if [ $(($1 % 50)) -eq 0 ]; then
		echo "**** Finished test $1 ****"
	fi
}

echo '****'
echo "Testing $NR_CPUS CPUs"
echo '****'

TEST=0
while :
do
	N=$((RANDOM % MAXCPU + 1))
	ON=`cat $(cpu_path $N)/online`
	echo $((1-ON)) > $(cpu_path $N)/online
	TEST=$((TEST+1))
	checkpoint_test $TEST
done