[PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

We really shouldn't be asking userspace to create new root filesystems.
So follow along with all of the other in-kernel filesystems, and provide
a mount point in sysfs.

For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
that mount point when the cgroup filesystem is registered in the kernel.

Cc: Paul Menage <menage@google.com>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Cc: Lennart Poettering <lennart@poettering.net>
Cc: Kay Sievers <kay.sievers@vrfy.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

--- a/kernel/cgroup.c
+++ b/kernel/cgroup.c
@@ -1623,6 +1623,8 @@ static struct file_system_type cgroup_fs_type = {
 	.kill_sb = cgroup_kill_sb,
 };
 
+static struct kobject *cgroup_kobj;
+
 static inline struct cgroup *__d_cgrp(struct dentry *dentry)
 {
 	return dentry->d_fsdata;
@@ -3871,9 +3873,18 @@ int __init cgroup_init(void)
 	hhead = css_set_hash(init_css_set.subsys);
 	hlist_add_head(&init_css_set.hlist, hhead);
 	BUG_ON(!init_root_id(&rootnode));
+
+	cgroup_kobj = kobject_create_and_add("cgroup", fs_kobj);
+	if (!cgroup_kobj) {
+		err = -ENOMEM;
+		goto out;
+	}
+
 	err = register_filesystem(&cgroup_fs_type);
-	if (err < 0)
+	if (err < 0) {
+		kobject_put(cgroup_kobj);
 		goto out;
+	}
 
 	proc_create("cgroups", 0, NULL, &proc_cgroupstats_operations);
 

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Paul Menage]

On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
> We really shouldn't be asking userspace to create new root filesystems.
> So follow along with all of the other in-kernel filesystems, and provide
> a mount point in sysfs.
>
> For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
> that mount point when the cgroup filesystem is registered in the kernel.

But cgroups will typically have multiple mounts, with different
resource controllers/options on each mount. That doesn't really fit in
with this scheme.

Paul

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
> On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
> > We really shouldn't be asking userspace to create new root filesystems.
> > So follow along with all of the other in-kernel filesystems, and provide
> > a mount point in sysfs.
> >
> > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
> > that mount point when the cgroup filesystem is registered in the kernel.
> 
> But cgroups will typically have multiple mounts, with different
> resource controllers/options on each mount. That doesn't really fit in
> with this scheme.

Really?  I see systems mounting it at /cgroups/ in the filesystem today.
Where are you expecting it to be mounted at?

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Paul Menage]

On Thu, Jul 22, 2010 at 11:36 AM, Greg KH <gregkh@suse.de> wrote:
>
> Really?  I see systems mounting it at /cgroups/ in the filesystem today.

Sure, it *can* be mounted as a single instance, but you then lose
flexibility. E.g. at Google we want to have a different hierarchy for
the CPU subsystem (with the tree grouped according to
latency-sensitive versus batch, etc) and memory (grouped according to
what jobs are sharing memory with each other).

> Where are you expecting it to be mounted at?

I have no particular expectation. (At Google we're actually using
/dev/cgroup/* but that's just for historical reasons, from
/dev/cpuset). Under /sys/fs/cgroup sounds reasonable, but you'd want
people to have the ability to manually create subdirs in there for
separate mount points.

Paul

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Thu, Jul 22, 2010 at 11:44:50AM -0700, Paul Menage wrote:
> On Thu, Jul 22, 2010 at 11:36 AM, Greg KH <gregkh@suse.de> wrote:
> >
> > Really?  I see systems mounting it at /cgroups/ in the filesystem today.
> 
> Sure, it *can* be mounted as a single instance, but you then lose
> flexibility. E.g. at Google we want to have a different hierarchy for
> the CPU subsystem (with the tree grouped according to
> latency-sensitive versus batch, etc) and memory (grouped according to
> what jobs are sharing memory with each other).
> 
> > Where are you expecting it to be mounted at?
> 
> I have no particular expectation. (At Google we're actually using
> /dev/cgroup/* but that's just for historical reasons, from
> /dev/cpuset). Under /sys/fs/cgroup sounds reasonable, but you'd want
> people to have the ability to manually create subdirs in there for
> separate mount points.

Ok, that's great, but it's not where the distros are starting to mount
it at.  As I learned with debugfs, you need to pick a location for
people to mount it at, otherwise it ends up all over the place.

If you are using /dev/cgroup/ that's nice, but I don't think that people
are expecting a whole filesystem under a /dev/ subdirectory.

I didn't realize that cgroupfs had so many different options that would
enable it to handle multiple mounts in this way.

Hm, Lennart, Kay, any ideas as to where to put it in a "standard" way?

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Paul Menage]

On Thu, Jul 22, 2010 at 11:53 AM, Greg KH <gregkh@suse.de> wrote:
>
> If you are using /dev/cgroup/ that's nice, but I don't think that people
> are expecting a whole filesystem under a /dev/ subdirectory.

Sure - as I said that's just for historical reasons from when we
migrated from cpusets which were traditionally mounted at /dev/cpuset,
so it involved less change. There's no particular reason to mount it
there (although having said that, what about /dev/pts and /dev/shm?).

Paul

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Thu, Jul 22, 2010 at 12:07:43PM -0700, Paul Menage wrote:
> On Thu, Jul 22, 2010 at 11:53 AM, Greg KH <gregkh@suse.de> wrote:
> >
> > If you are using /dev/cgroup/ that's nice, but I don't think that people
> > are expecting a whole filesystem under a /dev/ subdirectory.
> 
> Sure - as I said that's just for historical reasons from when we
> migrated from cpusets which were traditionally mounted at /dev/cpuset,
> so it involved less change. There's no particular reason to mount it
> there (although having said that, what about /dev/pts and /dev/shm?).

Yeah, /dev/pts and /dev/shm are long-time users of the /dev filesystem.
I don't know if we want to encourage that as a mount point, do you?

What are the different names you are giving to your mount points now for
cgroupfs so I can get an idea of how it is used currently?

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Paul Menage]

On Thu, Jul 22, 2010 at 12:12 PM, Greg KH <gregkh@suse.de> wrote:
> I don't know if we want to encourage that as a mount point, do you?

No.

>
> What are the different names you are giving to your mount points now for
> cgroupfs so I can get an idea of how it is used currently?
>

We don't particularly care about the names of the individual
sub-mounts, since it's all programatically controlled. We just specify
which subsystems we want grouped together (and a catch-all remainder
hierarchy) and the system picks the name of one of the subsystems in
the hierarchy to act as a unique mountpoint name (since each subsystem
can only be in one hierarchy). So e.g. /dev/cgroup/cpuset, or
/dev/cgroup/io

Paul

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Thu, Jul 22, 2010 at 12:18:13PM -0700, Paul Menage wrote:
> On Thu, Jul 22, 2010 at 12:12 PM, Greg KH <gregkh@suse.de> wrote:
> > I don't know if we want to encourage that as a mount point, do you?
> 
> No.

Heh.

> > What are the different names you are giving to your mount points now for
> > cgroupfs so I can get an idea of how it is used currently?
> >
> 
> We don't particularly care about the names of the individual
> sub-mounts, since it's all programatically controlled. We just specify
> which subsystems we want grouped together (and a catch-all remainder
> hierarchy) and the system picks the name of one of the subsystems in
> the hierarchy to act as a unique mountpoint name (since each subsystem
> can only be in one hierarchy). So e.g. /dev/cgroup/cpuset, or
> /dev/cgroup/io

Thanks for the info.  But in the end, I really don't want to see a
/cgroup/ mountpoint added to people's systems if at all possible.

Any suggestions from your side?

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Paul Menage]

On Thu, Jul 22, 2010 at 2:19 PM, Greg KH <gregkh@suse.de> wrote:
>
> Any suggestions from your side?
>

I'd probably go with Matt's suggestion of /var/run.

Paul

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Thu, Jul 22, 2010 at 04:12:58PM -0700, Paul Menage wrote:
> On Thu, Jul 22, 2010 at 2:19 PM, Greg KH <gregkh@suse.de> wrote:
> >
> > Any suggestions from your side?
> >
> 
> I'd probably go with Matt's suggestion of /var/run.

See Kay's answer for why this will not work.

sorry,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Matt Helsley]

On Thu, Jul 22, 2010 at 12:12:00PM -0700, Greg KH wrote:
> On Thu, Jul 22, 2010 at 12:07:43PM -0700, Paul Menage wrote:
> > On Thu, Jul 22, 2010 at 11:53 AM, Greg KH <gregkh@suse.de> wrote:
> > >
> > > If you are using /dev/cgroup/ that's nice, but I don't think that people
> > > are expecting a whole filesystem under a /dev/ subdirectory.
> > 
> > Sure - as I said that's just for historical reasons from when we
> > migrated from cpusets which were traditionally mounted at /dev/cpuset,
> > so it involved less change. There's no particular reason to mount it
> > there (although having said that, what about /dev/pts and /dev/shm?).
> 
> Yeah, /dev/pts and /dev/shm are long-time users of the /dev filesystem.
> I don't know if we want to encourage that as a mount point, do you?

Makes sense. cgroups don't share any of the distinctive characteristics
that make pts filesystems such a natural fit in /dev: They're
not made to look like a dynamic set of character devices, there are
user-made subdirectories, there aren't any device nodes, and probably
more I'm forgetting.

In my opinion, the primary purpose of cgroups is allowing an admin/user
to organize tasks in ways they personally find useful -- so too much
"standardization" could easily be detrimental. A single standard mount
point is the first small step on that slippery slope.

If we really want to standardize then perhaps it should be based on the
names of the tools that make assumptions about how cgroups are organized.
Forgive me if my recollection is wrong, but I seem to recall systemd
imposed a certain scheme for organization of cgroups -- at least at the
top level of the subsystem(s). (That's not a bad thing. Distros/admins
that choose to use systemd are implicitly approving of systemd's
organizational scheme and any limitations it imposes). So I'd suggest
those cgroup mounts go in /var/run/systemd/cgroup or something like that.

Suppose we also have a daemon for managing applications with the freezer to
improve runtime power consumption. It might only care about the freezer
cgroup subsystem so it might choose to mount the freezer at:

/var/run/runtime_powerd/freezer

and then it would organize its cgroups according to its own schemes
(e.g. "trusted" vs "untrusted").

But if systemd and the runtime_powerd both want the freezer subsystem
then one would fail to mount it. That ensures that we don't have two
management daemons which make contradictory assumptions about cgroup
organization making a mess instead of keeping order.

Other tools that are agnostic about the organization of cgroups
shouldn't rely on a 'standard' mount point -- that way they can work
well with either of these example organizations (systemd or runtime_powerd).
They should work with an existing cgroup and its descendants -- usually
not destroying pre-existing cgroups that it did not create.

At least that's my perspective on how things can be done sanely while
giving users the flexibility to manage their own cgroups. It's also why
I think the lure of standardizing these mounts may be something to be
wary of.

Cheers,
	-Matt Helsley

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Kay Sievers]

On Thu, Jul 22, 2010 at 23:48, Matt Helsley <matthltc@us.ibm.com> wrote:
> So I'd suggest
> those cgroup mounts go in /var/run/systemd/cgroup or something like that.

Impossible, we need it at bootup, and / is usually read-only, and /var
might even be behind the network we need to bring up.

It needs to be on tmpfs mounted on the rootsfs, or in any of the
virtual fss we have to mount anyway, so /sys/fs seems a good place.

Kay

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Matt Helsley]

On Fri, Jul 23, 2010 at 01:10:43AM +0200, Kay Sievers wrote:
> On Thu, Jul 22, 2010 at 23:48, Matt Helsley <matthltc@us.ibm.com> wrote:
> > So I'd suggest
> > those cgroup mounts go in /var/run/systemd/cgroup or something like that.
> 
> Impossible, we need it at bootup, and / is usually read-only, and /var
> might even be behind the network we need to bring up.
> 
> It needs to be on tmpfs mounted on the rootsfs, or in any of the
> virtual fss we have to mount anyway, so /sys/fs seems a good place.

Ahh, excellent point. Indeed, /sys/fs seems fine then.

Cheers,
	-Matt Helsley

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Vivek Goyal]

On Thu, Jul 22, 2010 at 11:36:15AM -0700, Greg KH wrote:
> On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
> > On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
> > > We really shouldn't be asking userspace to create new root filesystems.
> > > So follow along with all of the other in-kernel filesystems, and provide
> > > a mount point in sysfs.
> > >
> > > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
> > > that mount point when the cgroup filesystem is registered in the kernel.
> > 
> > But cgroups will typically have multiple mounts, with different
> > resource controllers/options on each mount. That doesn't really fit in
> > with this scheme.
> 
> Really?  I see systems mounting it at /cgroups/ in the filesystem today.
> Where are you expecting it to be mounted at?
> 

Greg,

[CCing few more folks who might be interested in this dicussion ]

We do want to retain facility to mount different controllers at different
mount points. We were discussing the other day that in libvirt we might
want to mount block IO controller and network controller separately as
by default we will not put a new virtual machine in a cgroup of its own
because of the penatly involved.

For other controllers like cpu, memory etc, libvirt automatically puts
each new virtual machine in a cgroup of own. So this is one use case
where we might want to mount different controllers at different mount
points.

For my testing I now always use /cgroup/ and create directories under it
/cgroup/blkio /cgroup/cpu etc and mount controllers on respective
directories.

Thanks
Vivek

> thanks,
> 
> greg k-h
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Thu, Jul 22, 2010 at 03:37:41PM -0400, Vivek Goyal wrote:
> On Thu, Jul 22, 2010 at 11:36:15AM -0700, Greg KH wrote:
> > On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
> > > On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
> > > > We really shouldn't be asking userspace to create new root filesystems.
> > > > So follow along with all of the other in-kernel filesystems, and provide
> > > > a mount point in sysfs.
> > > >
> > > > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
> > > > that mount point when the cgroup filesystem is registered in the kernel.
> > > 
> > > But cgroups will typically have multiple mounts, with different
> > > resource controllers/options on each mount. That doesn't really fit in
> > > with this scheme.
> > 
> > Really?  I see systems mounting it at /cgroups/ in the filesystem today.
> > Where are you expecting it to be mounted at?
> > 
> 
> Greg,
> 
> [CCing few more folks who might be interested in this dicussion ]
> 
> We do want to retain facility to mount different controllers at different
> mount points. We were discussing the other day that in libvirt we might
> want to mount block IO controller and network controller separately as
> by default we will not put a new virtual machine in a cgroup of its own
> because of the penatly involved.

That's fine, I'm not changing that ability at all.  We just need a
"default" mount point for "normal" users.

> For other controllers like cpu, memory etc, libvirt automatically puts
> each new virtual machine in a cgroup of own. So this is one use case
> where we might want to mount different controllers at different mount
> points.
> 
> For my testing I now always use /cgroup/ and create directories under it
> /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
> directories.

Lennart and Kay, is this what systemd is doing?  I really don't think we
should be adding a root /cgroup/ mount point to the system for something
like this.

Maybe /dev/cgroup/ is better to use, as that way users can create
sub-mount points easier.  They can't do that in /sys/fs/cgroup/

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Vivek Goyal]

On Thu, Jul 22, 2010 at 02:18:56PM -0700, Greg KH wrote:
> On Thu, Jul 22, 2010 at 03:37:41PM -0400, Vivek Goyal wrote:
> > On Thu, Jul 22, 2010 at 11:36:15AM -0700, Greg KH wrote:
> > > On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
> > > > On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
> > > > > We really shouldn't be asking userspace to create new root filesystems.
> > > > > So follow along with all of the other in-kernel filesystems, and provide
> > > > > a mount point in sysfs.
> > > > >
> > > > > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
> > > > > that mount point when the cgroup filesystem is registered in the kernel.
> > > > 
> > > > But cgroups will typically have multiple mounts, with different
> > > > resource controllers/options on each mount. That doesn't really fit in
> > > > with this scheme.
> > > 
> > > Really?  I see systems mounting it at /cgroups/ in the filesystem today.
> > > Where are you expecting it to be mounted at?
> > > 
> > 
> > Greg,
> > 
> > [CCing few more folks who might be interested in this dicussion ]
> > 
> > We do want to retain facility to mount different controllers at different
> > mount points. We were discussing the other day that in libvirt we might
> > want to mount block IO controller and network controller separately as
> > by default we will not put a new virtual machine in a cgroup of its own
> > because of the penatly involved.
> 
> That's fine, I'm not changing that ability at all.  We just need a
> "default" mount point for "normal" users.
> 
> > For other controllers like cpu, memory etc, libvirt automatically puts
> > each new virtual machine in a cgroup of own. So this is one use case
> > where we might want to mount different controllers at different mount
> > points.
> > 
> > For my testing I now always use /cgroup/ and create directories under it
> > /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
> > directories.
> 
> Lennart and Kay, is this what systemd is doing?  I really don't think we
> should be adding a root /cgroup/ mount point to the system for something
> like this.
> 
> Maybe /dev/cgroup/ is better to use, as that way users can create
> sub-mount points easier.  They can't do that in /sys/fs/cgroup/

The only problem with /dev/cgroup seems to be that it seems little
unintutive. To me, we have devices under /dev/ dir and cgroups are not
devices.

I think people have floated similar threads in the past on lkml with
various opinions and everybody had their own choices but nothing was
conclusive. 

Polluting / definitely sounds odd but it does not look that bad once
we can't find any other good choices.

Thanks
Vivek

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Thu, Jul 22, 2010 at 05:26:34PM -0400, Vivek Goyal wrote:
> On Thu, Jul 22, 2010 at 02:18:56PM -0700, Greg KH wrote:
> > On Thu, Jul 22, 2010 at 03:37:41PM -0400, Vivek Goyal wrote:
> > > On Thu, Jul 22, 2010 at 11:36:15AM -0700, Greg KH wrote:
> > > > On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
> > > > > On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
> > > > > > We really shouldn't be asking userspace to create new root filesystems.
> > > > > > So follow along with all of the other in-kernel filesystems, and provide
> > > > > > a mount point in sysfs.
> > > > > >
> > > > > > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
> > > > > > that mount point when the cgroup filesystem is registered in the kernel.
> > > > > 
> > > > > But cgroups will typically have multiple mounts, with different
> > > > > resource controllers/options on each mount. That doesn't really fit in
> > > > > with this scheme.
> > > > 
> > > > Really?  I see systems mounting it at /cgroups/ in the filesystem today.
> > > > Where are you expecting it to be mounted at?
> > > > 
> > > 
> > > Greg,
> > > 
> > > [CCing few more folks who might be interested in this dicussion ]
> > > 
> > > We do want to retain facility to mount different controllers at different
> > > mount points. We were discussing the other day that in libvirt we might
> > > want to mount block IO controller and network controller separately as
> > > by default we will not put a new virtual machine in a cgroup of its own
> > > because of the penatly involved.
> > 
> > That's fine, I'm not changing that ability at all.  We just need a
> > "default" mount point for "normal" users.
> > 
> > > For other controllers like cpu, memory etc, libvirt automatically puts
> > > each new virtual machine in a cgroup of own. So this is one use case
> > > where we might want to mount different controllers at different mount
> > > points.
> > > 
> > > For my testing I now always use /cgroup/ and create directories under it
> > > /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
> > > directories.
> > 
> > Lennart and Kay, is this what systemd is doing?  I really don't think we
> > should be adding a root /cgroup/ mount point to the system for something
> > like this.
> > 
> > Maybe /dev/cgroup/ is better to use, as that way users can create
> > sub-mount points easier.  They can't do that in /sys/fs/cgroup/
> 
> The only problem with /dev/cgroup seems to be that it seems little
> unintutive. To me, we have devices under /dev/ dir and cgroups are not
> devices.
> 
> I think people have floated similar threads in the past on lkml with
> various opinions and everybody had their own choices but nothing was
> conclusive. 
> 
> Polluting / definitely sounds odd but it does not look that bad once
> we can't find any other good choices.

I think /sys/fs/cgroup/ is a better choice than polluting /

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Balbir Singh]

* Vivek Goyal <vgoyal@redhat.com> [2010-07-22 17:26:34]:

> On Thu, Jul 22, 2010 at 02:18:56PM -0700, Greg KH wrote:
> > On Thu, Jul 22, 2010 at 03:37:41PM -0400, Vivek Goyal wrote:
> > > On Thu, Jul 22, 2010 at 11:36:15AM -0700, Greg KH wrote:
> > > > On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
> > > > > On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
> > > > > > We really shouldn't be asking userspace to create new root filesystems.
> > > > > > So follow along with all of the other in-kernel filesystems, and provide
> > > > > > a mount point in sysfs.
> > > > > >
> > > > > > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
> > > > > > that mount point when the cgroup filesystem is registered in the kernel.
> > > > > 
> > > > > But cgroups will typically have multiple mounts, with different
> > > > > resource controllers/options on each mount. That doesn't really fit in
> > > > > with this scheme.
> > > > 
> > > > Really?  I see systems mounting it at /cgroups/ in the filesystem today.
> > > > Where are you expecting it to be mounted at?
> > > > 
> > > 
> > > Greg,
> > > 
> > > [CCing few more folks who might be interested in this dicussion ]
> > > 
> > > We do want to retain facility to mount different controllers at different
> > > mount points. We were discussing the other day that in libvirt we might
> > > want to mount block IO controller and network controller separately as
> > > by default we will not put a new virtual machine in a cgroup of its own
> > > because of the penatly involved.
> > 
> > That's fine, I'm not changing that ability at all.  We just need a
> > "default" mount point for "normal" users.
> > 
> > > For other controllers like cpu, memory etc, libvirt automatically puts
> > > each new virtual machine in a cgroup of own. So this is one use case
> > > where we might want to mount different controllers at different mount
> > > points.
> > > 
> > > For my testing I now always use /cgroup/ and create directories under it
> > > /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
> > > directories.
> > 
> > Lennart and Kay, is this what systemd is doing?  I really don't think we
> > should be adding a root /cgroup/ mount point to the system for something
> > like this.
> > 
> > Maybe /dev/cgroup/ is better to use, as that way users can create
> > sub-mount points easier.  They can't do that in /sys/fs/cgroup/
> 
> The only problem with /dev/cgroup seems to be that it seems little
> unintutive. To me, we have devices under /dev/ dir and cgroups are not
> devices.
> 
> I think people have floated similar threads in the past on lkml with
> various opinions and everybody had their own choices but nothing was
> conclusive. 
> 
> Polluting / definitely sounds odd but it does not look that bad once
> we can't find any other good choices.
>

I think it breaks the filesystem hierarchy standard and I've seen
bugzilla's around it. I'd prefer /sys/fs/cgroup. 

-- 
	Three Cheers,
	Balbir

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Kay Sievers]

On Thu, Jul 22, 2010 at 23:18, Greg KH <gregkh@suse.de> wrote:
>> For my testing I now always use /cgroup/ and create directories under it
>> /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
>> directories.
>
> Lennart and Kay, is this what systemd is doing?  I really don't think we
> should be adding a root /cgroup/ mount point to the system for something
> like this.

Already solved. Systemd always mounts an empty tmpfs at the 'cgroup'
mountpoint, and stuff is free to create subdirs there. Systemd itself
mount 'systemd' there.

I guess that covers all needs expressed here, and we are happy to
switch systemd over from /cgroup to /sys/fs/cgroup as soon as the
kernel carries the patch.

Thanks,
Kay

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Fri, Jul 23, 2010 at 01:08:12AM +0200, Kay Sievers wrote:
> On Thu, Jul 22, 2010 at 23:18, Greg KH <gregkh@suse.de> wrote:
> >> For my testing I now always use /cgroup/ and create directories under it
> >> /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
> >> directories.
> >
> > Lennart and Kay, is this what systemd is doing?  I really don't think we
> > should be adding a root /cgroup/ mount point to the system for something
> > like this.
> 
> Already solved. Systemd always mounts an empty tmpfs at the 'cgroup'
> mountpoint, and stuff is free to create subdirs there. Systemd itself
> mount 'systemd' there.

Ah, that makes it easier.  Paul and Vivek, any objection to this patch
going in now?

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Paul Menage]

On Thu, Jul 22, 2010 at 4:14 PM, Greg KH <gregkh@suse.de> wrote:
>
> Ah, that makes it easier.  Paul and Vivek, any objection to this patch
> going in now?
>

Sounds reasonable.

Paul

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Kay Sievers]

On Thu, 2010-07-22 at 16:14 -0700, Greg KH wrote:
> On Fri, Jul 23, 2010 at 01:08:12AM +0200, Kay Sievers wrote:
> > On Thu, Jul 22, 2010 at 23:18, Greg KH <gregkh@suse.de> wrote:
> > >> For my testing I now always use /cgroup/ and create directories under it
> > >> /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
> > >> directories.
> > >
> > > Lennart and Kay, is this what systemd is doing?  I really don't think we
> > > should be adding a root /cgroup/ mount point to the system for something
> > > like this.
> > 
> > Already solved. Systemd always mounts an empty tmpfs at the 'cgroup'
> > mountpoint, and stuff is free to create subdirs there. Systemd itself
> > mount 'systemd' there.
> 
> Ah, that makes it easier.  Paul and Vivek, any objection to this patch
> going in now?

Just to get an idea, here is the current output on a systemd system:

/dev/sda3 /home/kay/work ext3 rw,noatime,errors=continue,user_xattr,acl,barrier=0,data=writeback 0 0
devtmpfs /dev devtmpfs rw,relatime,size=1965252k,nr_inodes=491313,mode=755 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,relatime 0 0
tmpfs /dev/shm tmpfs rw,relatime 0 0
devpts /dev/pts devpts rw,relatime,mode=600 0 0
tmpfs /cgroup tmpfs rw,nosuid,nodev,noexec,relatime,mode=755 0 0
cgroup /cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
systemd-1 /dev/mqueue autofs rw,relatime,fd=15,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
systemd-1 /sys/kernel/security autofs rw,relatime,fd=16,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=17,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
systemd-1 /sys/kernel/debug autofs rw,relatime,fd=18,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
systemd-1 /dev/hugepages autofs rw,relatime,fd=19,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
tmpfs /var/run tmpfs rw,relatime,mode=755 0 0
tmpfs /var/lock tmpfs rw,relatime,mode=775,gid=54 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
gvfs-fuse-daemon /home/kay/.gvfs fuse.gvfs-fuse-daemon rw,nosuid,nodev,relatime,user_id=2702,group_id=100 0 0

Kay

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Vivek Goyal]

On Thu, Jul 22, 2010 at 04:14:01PM -0700, Greg KH wrote:
> On Fri, Jul 23, 2010 at 01:08:12AM +0200, Kay Sievers wrote:
> > On Thu, Jul 22, 2010 at 23:18, Greg KH <gregkh@suse.de> wrote:
> > >> For my testing I now always use /cgroup/ and create directories under it
> > >> /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
> > >> directories.
> > >
> > > Lennart and Kay, is this what systemd is doing?  I really don't think we
> > > should be adding a root /cgroup/ mount point to the system for something
> > > like this.
> > 
> > Already solved. Systemd always mounts an empty tmpfs at the 'cgroup'
> > mountpoint, and stuff is free to create subdirs there. Systemd itself
> > mount 'systemd' there.
> 
> Ah, that makes it easier.  Paul and Vivek, any objection to this patch
> going in now?
> 

This sounds reasonable to me also.

Jan, I know you have been working in this area and raised concenrs about
cgroup mount point in the past. Does it look good to you?

Vivek

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Jan Safranek]

On 07/23/2010 04:07 AM, Vivek Goyal wrote:
> On Thu, Jul 22, 2010 at 04:14:01PM -0700, Greg KH wrote:
>> On Fri, Jul 23, 2010 at 01:08:12AM +0200, Kay Sievers wrote:
>>> On Thu, Jul 22, 2010 at 23:18, Greg KH<gregkh@suse.de>  wrote:
>>>>> For my testing I now always use /cgroup/ and create directories under it
>>>>> /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
>>>>> directories.
>>>>
>>>> Lennart and Kay, is this what systemd is doing?  I really don't think we
>>>> should be adding a root /cgroup/ mount point to the system for something
>>>> like this.
>>>
>>> Already solved. Systemd always mounts an empty tmpfs at the 'cgroup'
>>> mountpoint, and stuff is free to create subdirs there. Systemd itself
>>> mount 'systemd' there.
>>
>> Ah, that makes it easier.  Paul and Vivek, any objection to this patch
>> going in now?
>>
>
> This sounds reasonable to me also.
>
> Jan, I know you have been working in this area and raised concenrs about
> cgroup mount point in the past. Does it look good to you?

I would prefer some place where subdirectories can be created for 
individual hierarchies without additional tmpfs there. But as I 
understand /sys, it's not an easy feature to implement it there, so no, 
I am not against /sys/fs/cgroup.

Jan

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Fri, Jul 23, 2010 at 12:32:26PM +0200, Jan Safranek wrote:
> On 07/23/2010 04:07 AM, Vivek Goyal wrote:
>> On Thu, Jul 22, 2010 at 04:14:01PM -0700, Greg KH wrote:
>>> On Fri, Jul 23, 2010 at 01:08:12AM +0200, Kay Sievers wrote:
>>>> On Thu, Jul 22, 2010 at 23:18, Greg KH<gregkh@suse.de>  wrote:
>>>>>> For my testing I now always use /cgroup/ and create directories under it
>>>>>> /cgroup/blkio /cgroup/cpu etc and mount controllers on respective
>>>>>> directories.
>>>>>
>>>>> Lennart and Kay, is this what systemd is doing?  I really don't think we
>>>>> should be adding a root /cgroup/ mount point to the system for something
>>>>> like this.
>>>>
>>>> Already solved. Systemd always mounts an empty tmpfs at the 'cgroup'
>>>> mountpoint, and stuff is free to create subdirs there. Systemd itself
>>>> mount 'systemd' there.
>>>
>>> Ah, that makes it easier.  Paul and Vivek, any objection to this patch
>>> going in now?
>>>
>>
>> This sounds reasonable to me also.
>>
>> Jan, I know you have been working in this area and raised concenrs about
>> cgroup mount point in the past. Does it look good to you?
>
> I would prefer some place where subdirectories can be created for 
> individual hierarchies without additional tmpfs there. But as I understand 
> /sys, it's not an easy feature to implement it there, so no, I am not 
> against /sys/fs/cgroup.

Great, thanks for letting us know.

So, who can get this patch into mainline?

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Dhaval Giani]

On Thu, Jul 22, 2010 at 8:36 PM, Greg KH <gregkh@suse.de> wrote:
> On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
>> On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
>> > We really shouldn't be asking userspace to create new root filesystems.
>> > So follow along with all of the other in-kernel filesystems, and provide
>> > a mount point in sysfs.
>> >
>> > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
>> > that mount point when the cgroup filesystem is registered in the kernel.
>>
>> But cgroups will typically have multiple mounts, with different
>> resource controllers/options on each mount. That doesn't really fit in
>> with this scheme.
>
> Really?  I see systems mounting it at /cgroups/ in the filesystem today.
> Where are you expecting it to be mounted at?
>

Not really. It is getting mounted at /cgroups/<name of resource
controller>/ at a number of places. Keeping it in sysfs loses us a lot
of this flexibility. Unless you are ready to keep adding a new
mountpoint for each subsystem, it will not really work out in the long
term.

Dhaval

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Kay Sievers]

On Mon, Jul 26, 2010 at 11:08, Dhaval Giani <dhaval.lists@thegianis.in> wrote:
> On Thu, Jul 22, 2010 at 8:36 PM, Greg KH <gregkh@suse.de> wrote:
>> On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
>>> On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
>>> > We really shouldn't be asking userspace to create new root filesystems.
>>> > So follow along with all of the other in-kernel filesystems, and provide
>>> > a mount point in sysfs.
>>> >
>>> > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
>>> > that mount point when the cgroup filesystem is registered in the kernel.
>>>
>>> But cgroups will typically have multiple mounts, with different
>>> resource controllers/options on each mount. That doesn't really fit in
>>> with this scheme.
>>
>> Really?  I see systems mounting it at /cgroups/ in the filesystem today.
>> Where are you expecting it to be mounted at?
>>
>
> Not really. It is getting mounted at /cgroups/<name of resource
> controller>/ at a number of places. Keeping it in sysfs loses us a lot
> of this flexibility. Unless you are ready to keep adding a new
> mountpoint for each subsystem, it will not really work out in the long
> term.

As mentioned earlier in this thread, systemd already mounts a tmpfs at
the cgroup mountpoint. We need only a single directory. This should
not be an issue.

Kay

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Dhaval Giani]

On Mon, Jul 26, 2010 at 11:12 AM, Kay Sievers <kay.sievers@vrfy.org> wrote:
> On Mon, Jul 26, 2010 at 11:08, Dhaval Giani <dhaval.lists@thegianis.in> wrote:
>> On Thu, Jul 22, 2010 at 8:36 PM, Greg KH <gregkh@suse.de> wrote:
>>> On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
>>>> On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
>>>> > We really shouldn't be asking userspace to create new root filesystems.
>>>> > So follow along with all of the other in-kernel filesystems, and provide
>>>> > a mount point in sysfs.
>>>> >
>>>> > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
>>>> > that mount point when the cgroup filesystem is registered in the kernel.
>>>>
>>>> But cgroups will typically have multiple mounts, with different
>>>> resource controllers/options on each mount. That doesn't really fit in
>>>> with this scheme.
>>>
>>> Really?  I see systems mounting it at /cgroups/ in the filesystem today.
>>> Where are you expecting it to be mounted at?
>>>
>>
>> Not really. It is getting mounted at /cgroups/<name of resource
>> controller>/ at a number of places. Keeping it in sysfs loses us a lot
>> of this flexibility. Unless you are ready to keep adding a new
>> mountpoint for each subsystem, it will not really work out in the long
>> term.
>
> As mentioned earlier in this thread, systemd already mounts a tmpfs at
> the cgroup mountpoint. We need only a single directory. This should
> not be an issue.
>

Ah ok. I am catching up with email after over 3 weeks :-). Missed all
this discussion. My apologies!

Dhaval

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Mon, Jul 26, 2010 at 11:13:14AM +0200, Dhaval Giani wrote:
> On Mon, Jul 26, 2010 at 11:12 AM, Kay Sievers <kay.sievers@vrfy.org> wrote:
> > On Mon, Jul 26, 2010 at 11:08, Dhaval Giani <dhaval.lists@thegianis.in> wrote:
> >> On Thu, Jul 22, 2010 at 8:36 PM, Greg KH <gregkh@suse.de> wrote:
> >>> On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote:
> >>>> On Thu, Jul 22, 2010 at 11:26 AM, Greg KH <gregkh@suse.de> wrote:
> >>>> > We really shouldn't be asking userspace to create new root filesystems.
> >>>> > So follow along with all of the other in-kernel filesystems, and provide
> >>>> > a mount point in sysfs.
> >>>> >
> >>>> > For cgroupfs, this should be in /sys/fs/cgroup/  This change provides
> >>>> > that mount point when the cgroup filesystem is registered in the kernel.
> >>>>
> >>>> But cgroups will typically have multiple mounts, with different
> >>>> resource controllers/options on each mount. That doesn't really fit in
> >>>> with this scheme.
> >>>
> >>> Really?  I see systems mounting it at /cgroups/ in the filesystem today.
> >>> Where are you expecting it to be mounted at?
> >>>
> >>
> >> Not really. It is getting mounted at /cgroups/<name of resource
> >> controller>/ at a number of places. Keeping it in sysfs loses us a lot
> >> of this flexibility. Unless you are ready to keep adding a new
> >> mountpoint for each subsystem, it will not really work out in the long
> >> term.
> >
> > As mentioned earlier in this thread, systemd already mounts a tmpfs at
> > the cgroup mountpoint. We need only a single directory. This should
> > not be an issue.
> >
> 
> Ah ok. I am catching up with email after over 3 weeks :-). Missed all
> this discussion. My apologies!

Ok, again, after all of this, who is going to be applying this patch to
their tree for the .36 merge window?

Or should I apply it to my driver-core one?

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Paul Menage]

On Mon, Jul 26, 2010 at 2:28 PM, Greg KH <gregkh@suse.de> wrote:
>
> Ok, again, after all of this, who is going to be applying this patch to
> their tree for the .36 merge window?

There's no specific cgroups tree - cgroups patches generally go via akpm.

>
> Or should I apply it to my driver-core one?
>

That sounds like an OK option too.

Paul

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Mon, Jul 26, 2010 at 02:55:18PM -0700, Paul Menage wrote:
> On Mon, Jul 26, 2010 at 2:28 PM, Greg KH <gregkh@suse.de> wrote:
> >
> > Ok, again, after all of this, who is going to be applying this patch to
> > their tree for the .36 merge window?
> 
> There's no specific cgroups tree - cgroups patches generally go via akpm.
> 
> >
> > Or should I apply it to my driver-core one?
> >
> 
> That sounds like an OK option too.

Wonderful, can I get some acks from the cgroups maintainers?  I'll
gladly take it through my tree.

thanks,

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Dhaval Giani]

On Tue, Jul 27, 2010 at 12:05 AM, Greg KH <gregkh@suse.de> wrote:
> On Mon, Jul 26, 2010 at 02:55:18PM -0700, Paul Menage wrote:
>> On Mon, Jul 26, 2010 at 2:28 PM, Greg KH <gregkh@suse.de> wrote:
>> >
>> > Ok, again, after all of this, who is going to be applying this patch to
>> > their tree for the .36 merge window?
>>
>> There's no specific cgroups tree - cgroups patches generally go via akpm.
>>
>> >
>> > Or should I apply it to my driver-core one?
>> >
>>
>> That sounds like an OK option too.
>
> Wonderful, can I get some acks from the cgroups maintainers?  I'll
> gladly take it through my tree.
>

while I am not a maintainer, from the libcgroup side at least,

Acked-by: Dhaval Giani <dhaval.giani@gmail.com>

> thanks,
>
> greg k-h
>

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Tue, Jul 27, 2010 at 12:08:03AM +0200, Dhaval Giani wrote:
> On Tue, Jul 27, 2010 at 12:05 AM, Greg KH <gregkh@suse.de> wrote:
> > On Mon, Jul 26, 2010 at 02:55:18PM -0700, Paul Menage wrote:
> >> On Mon, Jul 26, 2010 at 2:28 PM, Greg KH <gregkh@suse.de> wrote:
> >> >
> >> > Ok, again, after all of this, who is going to be applying this patch to
> >> > their tree for the .36 merge window?
> >>
> >> There's no specific cgroups tree - cgroups patches generally go via akpm.
> >>
> >> >
> >> > Or should I apply it to my driver-core one?
> >> >
> >>
> >> That sounds like an OK option too.
> >
> > Wonderful, can I get some acks from the cgroups maintainers?  I'll
> > gladly take it through my tree.
> >
> 
> while I am not a maintainer, from the libcgroup side at least,
> 
> Acked-by: Dhaval Giani <dhaval.giani@gmail.com>

Thanks, I've added this to the patch.

greg k-h

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Paul Menage]

On Mon, Jul 26, 2010 at 3:05 PM, Greg KH <gregkh@suse.de> wrote:
>
> Wonderful, can I get some acks from the cgroups maintainers?  I'll
> gladly take it through my tree.

Sure.

Acked-by: Paul Menage <menage@google.com>

Paul

Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

[Greg KH]

On Mon, Jul 26, 2010 at 03:09:09PM -0700, Paul Menage wrote:
> On Mon, Jul 26, 2010 at 3:05 PM, Greg KH <gregkh@suse.de> wrote:
> >
> > Wonderful, can I get some acks from the cgroups maintainers?  I'll
> > gladly take it through my tree.
> 
> Sure.
> 
> Acked-by: Paul Menage <menage@google.com>

Wonderful, now queued up.

greg k-h