Re: [RFC][PATCH] sched: Start stopper early

[Oleg Nesterov <oleg@redhat.com>]

To avoid the confusion, let me repeat that I am not arguing with
this change, perhaps it makes sense too.

But unless I missed something it is not really correct and can't
fix the problem. So I still think the series I sent should be
applied first.

On 10/07, Peter Zijlstra wrote:
>  static int sched_cpu_active(struct notifier_block *nfb,
>  				      unsigned long action, void *hcpu)
>  {
> +	int cpu = (long)hcpu;
> +
>  	switch (action & ~CPU_TASKS_FROZEN) {
>  	case CPU_STARTING:
>  		set_cpu_rq_start_time();
>  		return NOTIFY_OK;
>  	case CPU_ONLINE:
> +		cpu_stopper_unpark(cpu);
>  		/*
>  		 * At this point a starting CPU has marked itself as online via
>  		 * set_cpu_online(). But it might not yet have marked itself
> @@ -5558,7 +5563,7 @@ static int sched_cpu_active(struct notifier_block *nfb,
>  		 * Thus, fall-through and help the starting CPU along.
>  		 */
>  	case CPU_DOWN_FAILED:
> -		set_cpu_active((long)hcpu, true);
> +		set_cpu_active(cpu, true);
>  		return NOTIFY_OK;
>  	default:
>  		return NOTIFY_DONE;
> diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
> index 12484e5..c674371 100644
> --- a/kernel/stop_machine.c
> +++ b/kernel/stop_machine.c
> @@ -496,6 +496,11 @@ static struct smp_hotplug_thread cpu_stop_threads = {
>  	.selfparking		= true,
>  };
>  
> +void cpu_stopper_unpark(unsigned int cpu)
> +{
> +	kthread_unpark(per_cpu(cpu_stopper.thread, cpu));
> +}

But note that kthread_unpark() will only wake the stopper thread up.

cpu_stopper->enabled is still false, and it will be false until
smpboot_unpark_thread() calls ->pre_unpark() later. And this means
that stop_two_cpus()->cpu_stop_queue_work() can silently fail until
then. So I don't this patch can fix the problem.

Oleg.