Re: [PATCH] Do not use cpu_to_node() to find an offlined cpu's node.

[Wen Congyang <wency@cn.fujitsu.com>]

At 10/09/2012 02:21 PM, David Rientjes Wrote:
> On Mon, 8 Oct 2012, Tang Chen wrote:
> 
>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> index 66b36ab..e76dce9 100644
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -1263,18 +1263,24 @@ EXPORT_SYMBOL_GPL(kick_process);
>>   */
>>  static int select_fallback_rq(int cpu, struct task_struct *p)
>>  {
>> -	const struct cpumask *nodemask = cpumask_of_node(cpu_to_node(cpu));
>> +	int nid = cpu_to_node(cpu);
>> +	const struct cpumask *nodemask = NULL;
>>  	enum { cpuset, possible, fail } state = cpuset;
>>  	int dest_cpu;
>>  
>> -	/* Look for allowed, online CPU in same node. */
>> -	for_each_cpu(dest_cpu, nodemask) {
>> -		if (!cpu_online(dest_cpu))
>> -			continue;
>> -		if (!cpu_active(dest_cpu))
>> -			continue;
>> -		if (cpumask_test_cpu(dest_cpu, tsk_cpus_allowed(p)))
>> -			return dest_cpu;
>> +	/* If the cpu has been offlined, its nid was set to -1. */
>> +	if (nid != -1) {
> 
> NUMA_NO_NODE.
> 
> Eek, the nid shouldn't be -1 yet, though, for cpu hotplug since this 
> should be called at CPU_DYING level and migrate_tasks() still sees a valid 
> cpu.

the cpu's node is set when the cpu is hotpluged(not online), and it will
be cleared when the cpu is hotremoved(This patch is in akpm tree):
https://lkml.org/lkml/2012/9/3/39

I guess the task is in sleep state when the cpu is offlined, and it doesn't
be migrated to another cpu.

Thanks
Wen Congyang

> 
> On x86, cpumask_of_node() is always guaranteed to return a valid cpumask 
> after boot so presumably this is a problem in some non-x86 arch code and 
> isn't actually a sched problem.
> 
>> +		nodemask = cpumask_of_node(nid);
>> +
>> +		/* Look for allowed, online CPU in same node. */
>> +		for_each_cpu(dest_cpu, nodemask) {
>> +			if (!cpu_online(dest_cpu))
>> +				continue;
>> +			if (!cpu_active(dest_cpu))
>> +				continue;
>> +			if (cpumask_test_cpu(dest_cpu, tsk_cpus_allowed(p)))
>> +				return dest_cpu;
>> +		}
>>  	}
>>  
>>  	for (;;) {
>