Re: [PATCH v3 1/7] sched: introduce primitives to account for CFS bandwidth tracking

[Bharata B Rao <bharata@linux.vnet.ibm.com>]

On Thu, Oct 14, 2010 at 06:08:34PM +0530, Balbir Singh wrote:
> * Peter Zijlstra <peterz@infradead.org> [2010-10-14 09:52:17]:
> 
> > On Wed, 2010-10-13 at 18:30 +0530, Balbir Singh wrote:
> > > > +static int tg_set_cfs_bandwidth(struct task_group *tg, u64 period, u64 quota)
> > > > +{
> > > > +	int i;
> > > > +	static DEFINE_MUTEX(mutex);
> > > > +
> > > > +	if (tg == &init_task_group)
> > > > +		return -EINVAL;
> > > > +
> > > > +	if (!period)
> > > > +		return -EINVAL;
> > > > +
> > > > +	/*
> > > > +	 * Ensure we have at least one tick of bandwidth every period.  This is
> > > > +	 * to prevent reaching a state of large arrears when throttled via
> > > > +	 * entity_tick() resulting in prolonged exit starvation.
> > > > +	 */
> > > > +	if (NS_TO_JIFFIES(quota) < 1)
> > > > +		return -EINVAL;
> > > 
> > > I hope we document this in the Documentation :)
> > 
> > /me went and looked up arrears in a dictionary and wonders why 'debt'
> > wasn't good enough.
> > > > +
> > > > +	mutex_lock(&mutex);
> > > > +	raw_spin_lock_irq(&tg->cfs_bandwidth.lock);
> > > > +	tg->cfs_bandwidth.period = ns_to_ktime(period);
> > > > +	tg->cfs_bandwidth.runtime = tg->cfs_bandwidth.quota = quota;
> > > > +	raw_spin_unlock_irq(&tg->cfs_bandwidth.lock);
> > > > +
> > > > +	for_each_possible_cpu(i) {
> > > 
> > > Why not for_each_online_cpu()?
> > 
> > Probably could be cured with a hotplug handler, but then you need to
> > track more state iirc.
> >
> 
> What more state? If a CPU is offline, we never get to it, do we? I
> think we need to do just an init and destroy - no?

Here we essentially initialize tg->cfs_rq[cpu]->quota_used{assigned}
for all CPUs. Given that we don't destroy tg->cfs_rq[cpu] and tg->se->[cpu]
when a CPU goes offline, is it really worth to have a notifier to just
initialize quota_used and quota_assigned when a CPU comes online ?

Regards,
Bharata.

> 
> > > > +		struct cfs_rq *cfs_rq = tg->cfs_rq[i];
> > > > +		struct rq *rq = rq_of(cfs_rq);
> > > > +
> > > > +		raw_spin_lock_irq(&rq->lock);
> > > > +		init_cfs_rq_quota(cfs_rq);
> > > > +		raw_spin_unlock_irq(&rq->lock);
> > > > +	}
> > > > +	mutex_unlock(&mutex);
> > > > +
> > > > +	return 0;
> > > > +}
> > 
> 
> -- 
> 	Three Cheers,
> 	Balbir