Re: [RFCv5 PATCH 32/46] sched: Energy-aware wake-up task placement

Morten Rasmussen
2015-07-20T15:35:40Z

On Thu, Jul 16, 2015 at 05:10:52PM -0700, Sai Gurrappadi wrote:
> Hi Morten,
>
> On 07/07/2015 11:24 AM, Morten Rasmussen wrote:
> > ---
>
> > +static int energy_aware_wake_cpu(struct task_struct *p, int target)
> > +{
> > + struct sched_domain *sd;
> > + struct sched_group *sg, *sg_target;
> > + int target_max_cap = INT_MAX;
> > + int target_cpu = task_cpu(p);
> > + int i;
> > +
> > + sd = rcu_dereference(per_cpu(sd_ea, task_cpu(p)));
> > +
> > + if (!sd)
> > + return target;
> > +
> > + sg = sd->groups;
> > + sg_target = sg;
> > +
> > + /*
> > + * Find group with sufficient capacity. We only get here if no cpu is
> > + * overutilized. We may end up overutilizing a cpu by adding the task,
> > + * but that should not be any worse than select_idle_sibling().
> > + * load_balance() should sort it out later as we get above the tipping
> > + * point.
> > + */
> > + do {
> > + /* Assuming all cpus are the same in group */
> > + int max_cap_cpu = group_first_cpu(sg);
> > +
> > + /*
> > + * Assume smaller max capacity means more energy-efficient.
> > + * Ideally we should query the energy model for the right
> > + * answer but it easily ends up in an exhaustive search.
> > + */
> > + if (capacity_of(max_cap_cpu) < target_max_cap &&
> > + task_fits_capacity(p, max_cap_cpu)) {
> > + sg_target = sg;
> > + target_max_cap = capacity_of(max_cap_cpu);
> > + }
> > + } while (sg = sg->next, sg != sd->groups);
>
> Should be capacity_orig_of(max_cap_cpu) right? Might select a suboptimal
> sg_target if max_cap_cpu has a significant amount of IRQ/RT activity.

Right, this heuristic isn't as good as I had hoped for.
task_fits_capacity() is using capacity_of() to check if we have
available capacity after subtracting RT/IRQ activity which should be
right but I only check the first cpu. So I might discard a group due to
RT/IRQ activity on cpu the first cpu while one the sibling cpus could be
fine. Then going for lowest capacity_of() means preferring group with
most RT/IRQ activity that still has enough capacity to fit the task.

Using capacity_orig_of() we would ignore RT/IRQ activity but is likely
to be better as we can try to avoid RQ/IRQ activity later. I will use
capacity_orig_of() here instead. Thanks.

> > +
> > + /* Find cpu with sufficient capacity */
> > + for_each_cpu_and(i, tsk_cpus_allowed(p), sched_group_cpus(sg_target)) {
> > + /*
> > + * p's blocked utilization is still accounted for on prev_cpu
> > + * so prev_cpu will receive a negative bias due the double
> > + * accouting. However, the blocked utilization may be zero.
> > + */
> > + int new_usage = get_cpu_usage(i) + task_utilization(p);
> > +
> > + if (new_usage > capacity_orig_of(i))
> > + continue;
>
> Is this supposed to be capacity_of(i) instead?

Yes, we should skip cpus with too much RT/IRQ activity here. Thanks.

Morten