Re: [patch 03/16] sched: introduce primitives to account for CFS bandwidth tracking

Paul Turner
2011-07-06T21:38:42Z

On Wed, Jun 22, 2011 at 3:52 AM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> On Tue, 2011-06-21 at 00:16 -0700, Paul Turner wrote:
>> +#ifdef CONFIG_CFS_BANDWIDTH
>> +       {
>> +               .name = "cfs_quota_us",
>> +               .read_s64 = cpu_cfs_quota_read_s64,
>> +               .write_s64 = cpu_cfs_quota_write_s64,
>> +       },
>> +       {
>> +               .name = "cfs_period_us",
>> +               .read_u64 = cpu_cfs_period_read_u64,
>> +               .write_u64 = cpu_cfs_period_write_u64,
>> +       },
>> +#endif
>
> Did I miss a reply to:
> lkml.kernel.org/r/1305538202.2466.4047.camel@twins ? why does it make
> sense to have different periods per cgroup? what does it mean?
>

Sorry for the delayed reply -- I never hit send on this one.

The reason asymmetric periods are beneficial is a trade-off exists
between latency and throughput. The 3 major "classes" I see are:

Latency sensitive applications with a very continuous work
distribution of work may look to use a very tight bandwidth period
(e.g. 10ms). This provides very consistent/predictable/repeatable
performance as well as limiting their bandwidth imposed tail
latencies.

Latency sensitive applications who experience "bursty", or
inconsistent work distributions. In this case expanding the period
slightly to improve burst capacity yields a large performance benefit;
while protecting the rest of the system's applications should they
burst beyond their provisioning.

Latency insensitive applications in which we care only about
throughput. For this type of application we care only about limiting
their usage over a prolonged period of time, with tail latency
concern. For applications in this class we can use large periods to
minimize overheads / maximize throughput.

These classes are somewhat orthogonal and as such they pack fairly
well on machines together; but support for this requires period
granularity to be at the hierarchy -- and not machine -- level.

(This is also briefly covered in the updated documentation.)