Re: [PATCH 1/2] cpuset: fix cpuset_cpus_allowed_fallback() don't update tsk->rt.nr_cpus_allowed

[Yong Zhang <yong.zhang0@gmail.com>]

On Sun, May 15, 2011 at 11:55:47AM -0700, Paul E. McKenney wrote:
> > Seems rcu_spawn_one_cpu_kthread() call wake_up_process() directly,
> > which is under hotplug event CPU_UP_PREPARE. Maybe it should be
> > under CPU_ONLINE.
> 
> Sorry, but this does not work.  The kthread must be running by the time

Yup, I also noticed that from the comments of rcu_spawn_one_cpu_kthread();

> the CPU appears, otherwise RCU grace periods in CPU_ONLINE notifiers
> will never complete.
> 
> This did turn out to be a scheduler bug -- see Cheng Xu's recent patch.
> (chengxu@linux.vnet.ibm.com)

Hmmm, I don't think they are same issue.

The patch sent by Cheng Xu is about rt runtime normalization during
CPU hotplug.

But this one is about too earlier rt_rq load balance.

Let consider this:

		on boot CPU
	...
	kernel_init();
	  smp_init();
	    _cpu_up();
	      __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
	        rcu_cpu_notify();
		  rcu_online_kthreads();
		    rcu_spawn_one_node_kthread();
		      wake_up_process();
		        try_to_wake_up();
			  select_task_rq();
			    select_fallback_rq();
			      cpuset_cpus_allowed_fallback();
			        /* here the rcu_thread's cpus_allowed will be set
				   to cpu_possible_mask, but now we only have the
				   boot cpu online, so it will run on the boot cpu
				   So if we have KOSAKI's patch applied,
				   p->rt.nr_cpus_allowed will be set to
				   cpumask_weight(cpu_possible_mask);
				 */
		      sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
		        __sched_setscheduler();
			  check_class_changed();
			    p->sched_class->switched_to(rq, p); /* rt_class */
			      push_rt_task();
			        find_lock_lowest_rq();
				  find_lowest_rq();
				    /* crash here because local_cpu_mask is uninitialized */


Thanks,
Yong