Re: [patch] Re: autogroup: sched_setscheduler() fails

[Mike Galbraith <efault@gmx.de>]

On Wed, 2011-01-12 at 09:07 +0530, Bharata B Rao wrote:
> On Tue, Jan 11, 2011 at 07:48:06PM +0100, Mike Galbraith wrote:
> > On Tue, 2011-01-11 at 22:40 +0530, Bharata B Rao wrote:
> > > On Mon, Jan 10, 2011 at 05:42:26PM +0100, Mike Galbraith wrote:
> > > > Index: linux-2.6/kernel/sched_autogroup.c
> > > > ===================================================================
> > > > --- linux-2.6.orig/kernel/sched_autogroup.c
> > > > +++ linux-2.6/kernel/sched_autogroup.c
> > > > @@ -73,6 +73,15 @@ static inline struct autogroup *autogrou
> > > >  	ag->id = atomic_inc_return(&autogroup_seq_nr);
> > > >  	ag->tg = tg;
> > > >  	tg->autogroup = ag;
> > > > +#ifdef CONFIG_RT_GROUP_SCHED
> > > > +	/*
> > > > +	 * HACK: autogroup RT tasks run in the root task group.
> > > > +	 * This fools __sched_setscheduler() into proceeding on
> > > > +	 * so we can move the task to the appropriate runqueue
> > > > +	 * upon scheduling policy change.
> > > > +	 */
> > > > +	tg->rt_bandwidth.rt_runtime = RUNTIME_INF;
> > > > +#endif
> > > > 
> > > >  	return ag;
> > > > 
> > > > @@ -143,6 +152,15 @@ autogroup_move_group(struct task_struct
> > > >  	autogroup_kref_put(prev);
> > > >  }
> > > > 
> > > > +static inline void
> > > > +autogroup_setscheduler(struct task_struct *p, int on_rq)
> > > > +{
> > > > +	if (p->sched_class->task_move_group)
> > > > +		p->sched_class->task_move_group(p, on_rq);
> > > > +	else
> > > > +		set_task_rq(p, task_cpu(p));
> > > > +}
> > > > +
> > > 
> > > IIUC, with the above changes you are actually queing the task into
> > > rt_rq of an autogroup. But the task's autogroup interface
> > > (/proc/<pid>/autogroup) allows you to control the bandwidth of only
> > > cfs_rq tasks, while the rt tasks in the group get RUNTIME_INF bandwidth.
> > > 
> > > I think what we need is a real group change here (which is difficult I
> > > think) and not just sched_class change.
> > 
> > Unless I fscked up, set_task_rq() is the group change.  As soon as the
> > task's class changes, it'll be moved to the root_task_group.
> 
> Ok, this is what I understand, may be Peter Z can confirm...
> 
> set_task_rq() just changes the task's cfs_rq and rt_rq as per its
> task_group().

Exactly.  When you call task_group() for an RT task, it returns
&root_task_group, so a task making the transition to RT will be dequeued
from it's autogroup cfs_rq, and enqueued in root_task_group's rt_rq.

>  The normal way to change a task's group is to first
> change its cgroup pointer (task->cgroups, see cgroup_attach_task())
> After this you change the runqueues by calling set_task_rq() which
> now refers to new group's runqueues.

Autogroup doesn't need to do cgroup stuff.  If a task is in a cgroup,
it's autogroup remains intact, but is ignored.  Move it back to root,
and it's autogroup will be used again.

	-Mike