Re: [RFC] [PATCH] cfq-iosched: add cfq group hierarchical scheduling support

[Vivek Goyal <vgoyal@redhat.com>]

On Wed, Sep 01, 2010 at 06:02:43PM +0900, KAMEZAWA Hiroyuki wrote:

[..]
> > > One of the possible way forward could be this.
> > > 
> > > - Treat queue and group at same level (like CFS)
> > > 
> > > - Get rid of cfq_slice_offset() logic. That means without idling on, there
> > >   will be no ioprio difference between cfq queues. I think anyway as of 
> > >   today that logic helps in so little situations that I would not mind
> > >   getting rid of it. Just that Jens should agree to it.
> > > 
> > > - With this new scheme, it will break the existing semantics of root group
> > >   being at same level as child groups. To avoid that, we can probably
> > >   implement two modes (flat and hierarchical), something similar to what
> > >   memory cgroup controller has done. May be one tunable in root cgroup of
> > >   blkio "use_hierarchy".  By default everything will be in flat mode and
> > >   if user wants hiearchical control, he needs to set user_hierarchy in
> > >   root group.
> > > 
> > >   I think memory controller provides "use_hierarchy" tunable in each
> > >   cgroup. I am not sure why do we need it in each cgroup and not just
> > >   in root cgroup.
> > 
> > I think Kamezawa-san should be able to answer this question. :)
> > 
> 
> At first, please be sure that "hierarchical accounting is _very_ slow".
> Please measure how hierarchical accounting (of 4-6 levels) are slow ;)
> 
> Then, there are 2 use cases.
> 
>  1)  root/to/some/directory/A
>                            /B
>                            /C
>                             ....
>      All A, B, C ....are flat cgroup and has no relationship, not sharing limit.
>      In this case, hierarchy should not be enabled.
> 
>  2)  root/to/some/directory/Gold/A,B,C...
>                             Silver/D,E,F
> 
>      All A, B, C ....are limited by "Gold" or "Silver".
>      But Gold and Silver has no relationthip, they has their own limitations.
>      But A, B, C, D, E, F shares limit under Gold or Silver.
>      In this case, hierarchy
>      "root/to/some/directory" should be disabled.
>       Gold/ and Silver should have use_hierarchy=1.
> 
> (Assume these Gold and Silver as Container and the user of container
>  divides memory into A and B, C...)
> 
> For example,  libvirt makes very long "root/to/some/directory" ...
> I never want to count-up all counters in the hierarchy even if
> we'd like to use some fantasic hierarchical accounting under a container.
> 
> I don't like "all or nothing" option (as making use_hierarchy as mount
> option or has parameter on root cgroup etc..) Then, allowed mixture.

Hi Kame San,

If you don't want any relationship between Gold and Silver then one can
make root as unlimited group (limit_in_bytes = -1) and practically Gold
and Silver have no dependency. There is no need of setting use_hierarchy
different at root level and inside Gold/ and Silver/ groups?

It sounds like you did it for two reasons.

- It can potentially provide more flexibility. 
- performance reason so that you can stop do hierarchical accounting
  all the way to root and stop before that (libvirt example).

I think for blkio controller we can probably begin with either a mount
time option or a use_hierachy file in root group and then later make
it per group if there are use cases.

Vivek