Re: [PATCH 2/6] sched: make sched_slice() group scheduling savvy

Srivatsa Vaddagiri
2007-11-01T16:19:05Z

On Thu, Nov 01, 2007 at 01:20:08PM +0100, Peter Zijlstra wrote:
> On Thu, 2007-11-01 at 13:03 +0100, Peter Zijlstra wrote:
> > On Thu, 2007-11-01 at 12:58 +0100, Peter Zijlstra wrote:
> >
> > > > sched_slice() is about lantecy, its intended purpose is to ensure each
> > > > task is ran exactly once during sched_period() - which is
> > > > sysctl_sched_latency when nr_running <= sysctl_sched_nr_latency, and
> > > > otherwise linearly scales latency.
> >
> > The thing that got my brain in a twist is what to do about the non-leaf
> > nodes, for those it seems I'm not doing the right thing - I think.
>
> Ok, suppose a tree like so:
>
>
> level 2 cfs_rq
> A B
>
> level 1 cfs_rqA cfs_rqB
> A0 B0 - B99
>
>
> So for sake of determinism, we want to impose a period in which all
> level 1 tasks will have ran (at least) once.

Peter,
I fail to see why this requirement to "determine a period in
which all level 1 tasks will have ran (at least) once" is essential.

I am visualizing each of the groups to be similar to Xen-like partitions
which are given fair timeslices by the hypervisor (Linux kernel in this
case). How each partition (group in this case) manages the allocated
timeslice(s) to provide fairness to tasks within that partition/group should not
(IMHO) depend on other groups and esp. how many tasks other groups has.

For ex: before this patch, fair time would be allocated to group and
their tasks as below:

A0 B0-B9 A0 B10-B19 A0 B20-B29
|--------|--------|--------|--------|--------|--------|-----//--|
0 10ms 20ms 30ms 40ms 50ms 60ms

i.e during the first 10ms allocated to group B, B0-B9 run,
during the next 10ms allocated to group B, B10-B19 run etc

What's wrong with this scheme?

By letting __sched_period() be determined for each group independently,
we are building stronger isolation between them, which is good IMO
(imagine a rogue container that does a fork bomb).

> Now what sched_slice() does is calculate the weighted proportion of the
> given period for each task to run, so that each task runs exactly once.
>
> Now level 2, can introduce these large weight differences, which in this
> case result in 'lumps' of time.
>
> In the given example above the weight difference is 1:100, which is
> already at the edges of what regular nice levels could do.
>
> How about limiting the max output of sched_slice() to
> sysctl_sched_latency in order to break up these large stretches of time?
>
> Index: linux-2.6/kernel/sched_fair.c
> ===================================================================
> --- linux-2.6.orig/kernel/sched_fair.c
> +++ linux-2.6/kernel/sched_fair.c
> @@ -341,7 +341,7 @@ static u64 sched_slice(struct cfs_rq *cf
> do_div(slice, cfs_rq->load.weight);
> }
>
> - return slice;
> + return min_t(u64, sysctl_sched_latency, slice);

Hmm, going back to the previous example I cited, this will lead to:

sched_slice(grp A) = min(20ms, 500ms) = 20ms
sched_slice(A0) = min(20ms, 500ms) = 20ms

sched_slice(grp B) = min(20ms, 500ms) = 20ms
sched_slice(B0) = min(20ms, 0.5ms) = 0.5ms

Fairness between groups and tasks would be obtained as below:

A0 B0-B39 A0 B40-B79 A0
|--------|--------|--------|--------|--------|
0 20ms 40ms 60ms 80ms

which seems to be more or less giving what we already have w/o the
patch?

--
Regards,
vatsa