Re: [PATCH] sched: unthrottle rt_rq in migrate_tasks()

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

On Fri, 2012-05-18 at 11:56 -0700, Peter Boonstoppel wrote:
> migrate_tasks() uses _pick_next_task_rt() to get tasks from the
> real-time runqueue to be migrated. When rt_rq is throttled
> _pick_next_task_rt() won't return anything, in which case
> migrate_tasks() can't move all threads over and gets stuck in an
> infinite loop.
> 
> Instead unthrottle rt_rq before migrating tasks.
> 
> Signed-off-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
> ---
>  kernel/sched/core.c |    6 ++++++
>  1 files changed, 6 insertions(+), 0 deletions(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 9236a48..b37e0b3 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -5159,6 +5159,12 @@ static void migrate_tasks(unsigned int dead_cpu)
>  	/* Ensure any throttled groups are reachable by pick_next_task */
>  	unthrottle_offline_cfs_rqs(rq);
>  
> +	/*
> +	 * Ensure rt_rq is not throttled so its threads can be migrated using
> +	 * pick_next_task_rt
> +	 */
> +	rq->rt.rt_throttled = 0;
> +
>  	for ( ; ; ) {
>  		/*
>  		 * There's this thread running, bail when that's the only

OK, so there's two things:

 - there's a set_rq_offline() call in CPU_DYING right before calling
migrate_tasks(). This would end up calling rq_offline_rt() which should
disable all the runtime muck.

 - If the above doesn't unthrottle rqs (looks like it doesn't, should
it?) you still only unthrottle the root and 'forgot' to deal with
cgroups.

So I think I'm going to ask you to make rq_offline_rt() dtrt, and while
you're at it, move unthrottle_offline_cfs_rqs() to rq_offline_fair().