Re: [patch 03/16] sched: introduce primitives to account for CFS bandwidth tracking

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

On Wed, 2011-07-06 at 14:38 -0700, Paul Turner wrote:
> On Wed, Jun 22, 2011 at 3:52 AM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> > On Tue, 2011-06-21 at 00:16 -0700, Paul Turner wrote:
> >> +#ifdef CONFIG_CFS_BANDWIDTH
> >> +       {
> >> +               .name = "cfs_quota_us",
> >> +               .read_s64 = cpu_cfs_quota_read_s64,
> >> +               .write_s64 = cpu_cfs_quota_write_s64,
> >> +       },
> >> +       {
> >> +               .name = "cfs_period_us",
> >> +               .read_u64 = cpu_cfs_period_read_u64,
> >> +               .write_u64 = cpu_cfs_period_write_u64,
> >> +       },
> >> +#endif
> >
> > Did I miss a reply to:
> > lkml.kernel.org/r/1305538202.2466.4047.camel@twins ? why does it make
> > sense to have different periods per cgroup? what does it mean?
> >
> 
> Sorry for the delayed reply -- I never hit send on this one.
> 
> The reason asymmetric periods are beneficial is a trade-off exists
> between latency and throughput.  The 3 major "classes" I see are:
> 
> Latency sensitive applications with a very continuous work
> distribution of work may look to use a very tight bandwidth period
> (e.g. 10ms).  This provides very consistent/predictable/repeatable
> performance as well as limiting their bandwidth imposed tail
> latencies.
> 
> Latency sensitive applications who experience "bursty", or
> inconsistent work distributions.  In this case expanding the period
> slightly to improve burst capacity yields a large performance benefit;
> while protecting the rest of the system's applications should they
> burst beyond their provisioning.
> 
> Latency insensitive applications in which we care only about
> throughput.  For this type of application we care only about limiting
> their usage over a prolonged period of time, with tail latency
> concern.  For applications in this class we can use large periods to
> minimize overheads / maximize throughput.
> 
> These classes are somewhat orthogonal and as such they pack fairly
> well on machines together; but support for this requires period
> granularity to be at the hierarchy -- and not machine -- level.
> 
> (This is also briefly covered in the updated documentation.)


Right, but what I'm getting at is the hierarchical nature of things.
Having a parent/child both both with bandwidth constraints but different
periods simply doesn't work well. The child will always be subjected to
the parent's sleep time and thus affected by its choice of period.

One way out of there is not allowing child cgroups to set their period
when they have a parent that is also bandwidth constrained, and have the
period update of a cgroup without constrained parent update its entire
sub-tree.

That way you can still have siblings with different periods as children
of an unconstrained parent (say root), but avoid all the weirdness of
parent/child with different constraint periods.