Re: [RFC][PATCH 00/11] blkiocg async support

[Vivek Goyal <vgoyal@redhat.com>]

On Thu, Jul 15, 2010 at 09:00:48AM +0900, KAMEZAWA Hiroyuki wrote:
> On Wed, 14 Jul 2010 10:29:19 -0400
> Vivek Goyal <vgoyal@redhat.com> wrote:
> > > 
> > > Cgroup's feature as mounting several subsystems at a mount point at once
> > > is very useful in many case.
> > 
> > I agree that it is useful but if some controllers are not supporting
> > hierarchy, it just adds to more confusion. And later when hierarchy
> > support comes in, there will be additional issue of keeping this file
> > "use_hierarchy" like memory controller.
> > 
> > So at this point of time , I am not too inclined towards allowing hierarchical
> > cgroup creation but treating them as flat in CFQ. I think it adds to the
> > confusion and user space should handle this situation.
> > 
> 
> Hmm. 
> 
> Could you fix error code in create blkio cgroup ? It returns -EINVAL now.
> IIUC, mkdir(2) doesn't return -EINVAL as error code (from man.)
> Then, it's very confusing. I think -EPERM or -ENOMEM will be much better.

Hm..., Probably -EPERM is somewhat close to what we are doing. File system
does supoort creation of directories but not after certain level.

I will trace more instances of mkdir error values.

> 
> Anyway, I need to see source code of blk-cgroup.c to know why libvirt fails
> to create cgroup.

[CCing daniel berrange]

AFAIK, libvirt does not have support for blkio controller yet. Are you 
trying to introduce that? 

libvirt creates a direcotry tree. I think /cgroup/libvirt/qemu/kvm-dirs.
So actual virtual machine directors are 2-3 level below and that would
explain that if you try to use blkio controller with libvirt, it will fail
because it will not be able to create directories at that level.

I think libvirt need to special case blkio here to create directories in 
top level. It is odd but really there are no easy answeres. Will we not
support a controller in libvirt till controller support hierarchy.

> Where is the user-visible information (in RHEL or Fedora)
> about "you can't use blkio-cgroup via libvirt or libcgroup" ?

[CCing balbir]

I think with libcgroup you can use blkio controller. I know somebody
who was using cgexec command to launch some jobs in blkio cgroups. AFAIK,
libcgroup does not have too much controller specific state and should
not require any modifications for blkio controller. 

Balbir can tell us more.

libvirt will require modification to support blkio controller. I also 
noticed that libvirt by default puts every virtual machine into its
own cgroup. I think it might not be a very good strategy for blkio
controller because putting every virtual machine in its own cgroup
will kill overall throughput if each virtual machine is not driving
enough IO.

I am also trying to come up with some additional logic of letting go 
fairness if a group is not doing sufficient IO.

Daniel, do you know where is the documentation which says what controllers
are currently supported by libvirt.

Thanks
Vivek