Re: [tip:sched/core] sched: Migrate waking tasks

[Kirill Tkhai <tkhai@yandex.ru>]

09.09.2014, 18:54, "tip-bot for Lai Jiangshan" <tipbot@zytor.com>:
> Commit-ID:  5cd038f53ed9ec7a17ab7d536a727363080f4210
> Gitweb:     http://git.kernel.org/tip/5cd038f53ed9ec7a17ab7d536a727363080f4210
> Author:     Lai Jiangshan <laijs@cn.fujitsu.com>
> AuthorDate: Wed, 4 Jun 2014 16:25:15 +0800
> Committer:  Ingo Molnar <mingo@kernel.org>
> CommitDate: Tue, 9 Sep 2014 06:47:27 +0200
>
> sched: Migrate waking tasks
>
> Current code can fail to migrate a waking task (silently) when TTWU_QUEUE is
> enabled.
>
> When a task is waking, it is pending on the wake_list of the rq, but it is not
> queued (task->on_rq == 0). In this case, set_cpus_allowed_ptr() and
> __migrate_task() will not migrate it because its invisible to them.
>
> This behavior is incorrect, because the task has been already woken, it will be
> running on the wrong CPU without correct placement until the next wake-up or
> update for cpus_allowed.
>
> To fix this problem, we need to finish the wakeup (so they appear on
> the runqueue) before we migrate them.
>
> Reported-by: Sasha Levin <sasha.levin@oracle.com>
> Reported-by: Jason J. Herne <jjherne@linux.vnet.ibm.com>
> Tested-by: Jason J. Herne <jjherne@linux.vnet.ibm.com>
> Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Link: http://lkml.kernel.org/r/538ED7EB.5050303@cn.fujitsu.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> ---
>  kernel/sched/core.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index a814b3c..78e5c83 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4666,7 +4666,7 @@ int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask)
>  		goto out;
>
>	dest_cpu = cpumask_any_and(cpu_active_mask, new_mask);
> -	if (task_on_rq_queued(p)) {
> +	if (task_on_rq_queued(p) || p->state == TASK_WAKING) {
>  		struct migration_arg arg = { p, dest_cpu };
>  		/* Need help from migration thread: drop lock and wait. */
>  		task_rq_unlock(rq, p, &flags);


About migration_cpu_stop():

> @@ -4799,6 +4799,12 @@ static int migration_cpu_stop(void *data)
>  	* be on another cpu but it doesn't matter.
>  	*/
>  	local_irq_disable();
> +	/*
> +	* We need to explicitly wake pending tasks before running
> +	* __migrate_task() such that we will not miss enforcing cpus_allowed
> +	* during wakeups, see set_cpus_allowed_ptr()'s TASK_WAKING test.
> +	*/
> +	sched_ttwu_pending();
>	__migrate_task(arg->task, raw_smp_processor_id(), arg->dest_cpu);
>	local_irq_enable();
>	return 0;

It looks like we do not need this hunk, because IPI happens earlier then
stop class begins migration_cpu_stop() execution.

In the first hunk the check "p->state == TASK_WAKING" is under pi_lock,
so if the task is really waking then the IPI is already set.

So, the first hunk is enough here, the second is not need.

Kirill

P.S. Formatting may be in punk style