[PATCH] cpuset: Allow cpuset controller in default hierarchy

[PATCH] cpuset: Allow cpuset controller in default hierarchy

[Waiman Long]

Given the fact that thread mode is now queued for 4.14, it is now
time to enable cpuset to be used in the default hierarchy (cgroup v2)
as it is clearly threaded.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index f3539a4..fb99d66 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2100,7 +2100,9 @@ struct cgroup_subsys cpuset_cgrp_subsys = {
 	.bind		= cpuset_bind,
 	.fork		= cpuset_fork,
 	.legacy_cftypes	= files,
+	.dfl_cftypes	= files,
 	.early_init	= true,
+	.threaded	= true,
 };
 
 /**
-- 
1.8.3.1

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Tejun Heo]

Hello, Waiman.

On Thu, Aug 24, 2017 at 10:40:11AM -0400, Waiman Long wrote:
> Given the fact that thread mode is now queued for 4.14, it is now
> time to enable cpuset to be used in the default hierarchy (cgroup v2)
> as it is clearly threaded.

I'm not sure we want things like exclusive settings in v2 as it
directly violates basic hierarchy rules (a sibling shouldn't be able
to affect what another sibling can or can't do).  We need to think
more about what should be exposed in v2.

Thanks.

-- 
tejun

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Waiman Long]

On 08/24/2017 10:43 AM, Tejun Heo wrote:
> Hello, Waiman.
>
> On Thu, Aug 24, 2017 at 10:40:11AM -0400, Waiman Long wrote:
>> Given the fact that thread mode is now queued for 4.14, it is now
>> time to enable cpuset to be used in the default hierarchy (cgroup v2)
>> as it is clearly threaded.
> I'm not sure we want things like exclusive settings in v2 as it
> directly violates basic hierarchy rules (a sibling shouldn't be able
> to affect what another sibling can or can't do).  We need to think
> more about what should be exposed in v2.

Thank for the quick response.

The main reason for sending out this patch is to figure out what exactly
is in your mind before enabling cpuset in v2.

From my point of view, the exclusive setting makes cpuset behave more
like a resource domain that a resource allocated to one cgroup won't be
available for another cgroup. So we can argue it both ways whether it
violates the basic hierarchy rules or not.

I will be in the Plumbers Conference next month and we can talk more
about this. My goal is to make cgroup v2 ready for prime time hopefully
by the end of the year.

Cheers,
Longman

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Waiman Long]

On 08/24/2017 10:43 AM, Tejun Heo wrote:
> Hello, Waiman.
>
> On Thu, Aug 24, 2017 at 10:40:11AM -0400, Waiman Long wrote:
>> Given the fact that thread mode is now queued for 4.14, it is now
>> time to enable cpuset to be used in the default hierarchy (cgroup v2)
>> as it is clearly threaded.
> I'm not sure we want things like exclusive settings in v2 as it
> directly violates basic hierarchy rules (a sibling shouldn't be able
> to affect what another sibling can or can't do).  We need to think
> more about what should be exposed in v2.

Thank for the quick response.

The main reason for sending out this patch is to figure out what exactly
is in your mind before enabling cpuset in v2.

>From my point of view, the exclusive setting makes cpuset behave more
like a resource domain that a resource allocated to one cgroup won't be
available for another cgroup. So we can argue it both ways whether it
violates the basic hierarchy rules or not.

I will be in the Plumbers Conference next month and we can talk more
about this. My goal is to make cgroup v2 ready for prime time hopefully
by the end of the year.

Cheers,
Longman

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Tejun Heo]

Hello,

On Thu, Aug 24, 2017 at 10:56:21AM -0400, Waiman Long wrote:
> The main reason for sending out this patch is to figure out what exactly
> is in your mind before enabling cpuset in v2.
> 
> From my point of view, the exclusive setting makes cpuset behave more
> like a resource domain that a resource allocated to one cgroup won't be
> available for another cgroup. So we can argue it both ways whether it
> violates the basic hierarchy rules or not.

It's not even siblings interfering with other siblings.  The resource
knobs belong to the parent, right?  So, the only thing it's doing is
restricting the parent itself from creating certain configurations,
which can be argued to be a feature but it's a really weird policy
enforcement implemented in kernel.

> I will be in the Plumbers Conference next month and we can talk more
> about this. My goal is to make cgroup v2 ready for prime time hopefully
> by the end of the year.

Sure, the only thing I think we need for cpuset is cutting down the
interface to the minimal set which provides all the features to
userspace.

Thanks.

-- 
tejun

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Tejun Heo]

Hello,

On Thu, Aug 24, 2017 at 10:56:21AM -0400, Waiman Long wrote:
> The main reason for sending out this patch is to figure out what exactly
> is in your mind before enabling cpuset in v2.
> 
> From my point of view, the exclusive setting makes cpuset behave more
> like a resource domain that a resource allocated to one cgroup won't be
> available for another cgroup. So we can argue it both ways whether it
> violates the basic hierarchy rules or not.

It's not even siblings interfering with other siblings.  The resource
knobs belong to the parent, right?  So, the only thing it's doing is
restricting the parent itself from creating certain configurations,
which can be argued to be a feature but it's a really weird policy
enforcement implemented in kernel.

> I will be in the Plumbers Conference next month and we can talk more
> about this. My goal is to make cgroup v2 ready for prime time hopefully
> by the end of the year.

Sure, the only thing I think we need for cpuset is cutting down the
interface to the minimal set which provides all the features to
userspace.

Thanks.

-- 
tejun

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Waiman Long]

On 08/24/2017 11:02 AM, Tejun Heo wrote:
> Hello,
>
> On Thu, Aug 24, 2017 at 10:56:21AM -0400, Waiman Long wrote:
>> The main reason for sending out this patch is to figure out what exactly
>> is in your mind before enabling cpuset in v2.
>>
>> From my point of view, the exclusive setting makes cpuset behave more
>> like a resource domain that a resource allocated to one cgroup won't be
>> available for another cgroup. So we can argue it both ways whether it
>> violates the basic hierarchy rules or not.
> It's not even siblings interfering with other siblings.  The resource
> knobs belong to the parent, right?  So, the only thing it's doing is
> restricting the parent itself from creating certain configurations,
> which can be argued to be a feature but it's a really weird policy
> enforcement implemented in kernel.

Yes, it is an unusual feature. It is basically an enforcement in kernel
of what the user space can perfectly do by itself without the
involvement by the kernel. Perhaps, we should leave it out from v2 for
now. We can always add it back when the developers ask for it, but
deleting feature once it is in is a no-no.

>> I will be in the Plumbers Conference next month and we can talk more
>> about this. My goal is to make cgroup v2 ready for prime time hopefully
>> by the end of the year.
> Sure, the only thing I think we need for cpuset is cutting down the
> interface to the minimal set which provides all the features to
> userspace.

Besides *cpus and *mems, the rests are just flags that can be turned on
or off. Maybe we could use a single cpuset.flags file to replace all
those files and use + and - prefix to turn on and off flags like
subtree_control. That should reduce the interface bloat which can be a
concern if a large number of controllers are enabled in a cgroup.

Cheers,
Longman

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Waiman Long]

On 08/24/2017 11:02 AM, Tejun Heo wrote:
> Hello,
>
> On Thu, Aug 24, 2017 at 10:56:21AM -0400, Waiman Long wrote:
>> The main reason for sending out this patch is to figure out what exactly
>> is in your mind before enabling cpuset in v2.
>>
>> From my point of view, the exclusive setting makes cpuset behave more
>> like a resource domain that a resource allocated to one cgroup won't be
>> available for another cgroup. So we can argue it both ways whether it
>> violates the basic hierarchy rules or not.
> It's not even siblings interfering with other siblings.  The resource
> knobs belong to the parent, right?  So, the only thing it's doing is
> restricting the parent itself from creating certain configurations,
> which can be argued to be a feature but it's a really weird policy
> enforcement implemented in kernel.

Yes, it is an unusual feature. It is basically an enforcement in kernel
of what the user space can perfectly do by itself without the
involvement by the kernel. Perhaps, we should leave it out from v2 for
now. We can always add it back when the developers ask for it, but
deleting feature once it is in is a no-no.

>> I will be in the Plumbers Conference next month and we can talk more
>> about this. My goal is to make cgroup v2 ready for prime time hopefully
>> by the end of the year.
> Sure, the only thing I think we need for cpuset is cutting down the
> interface to the minimal set which provides all the features to
> userspace.

Besides *cpus and *mems, the rests are just flags that can be turned on
or off. Maybe we could use a single cpuset.flags file to replace all
those files and use + and - prefix to turn on and off flags like
subtree_control. That should reduce the interface bloat which can be a
concern if a large number of controllers are enabled in a cgroup.

Cheers,
Longman

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Christian Brauner]

On Thu, Aug 24, 2017 at 08:02:22AM -0700, Tejun Heo wrote:
> Hello,
> 
> On Thu, Aug 24, 2017 at 10:56:21AM -0400, Waiman Long wrote:
> > The main reason for sending out this patch is to figure out what exactly
> > is in your mind before enabling cpuset in v2.
> > 
> > From my point of view, the exclusive setting makes cpuset behave more
> > like a resource domain that a resource allocated to one cgroup won't be
> > available for another cgroup. So we can argue it both ways whether it
> > violates the basic hierarchy rules or not.
> 
> It's not even siblings interfering with other siblings.  The resource
> knobs belong to the parent, right?  So, the only thing it's doing is
> restricting the parent itself from creating certain configurations,
> which can be argued to be a feature but it's a really weird policy
> enforcement implemented in kernel.
> 
> > I will be in the Plumbers Conference next month and we can talk more
> > about this. My goal is to make cgroup v2 ready for prime time hopefully
> > by the end of the year.

Hi,

I've been following this thread and the cpuset controller being enabled on
cgroup v2 is a really important step. As one of many container runtime
maintainers this is obviously a big step for us with a lot of impact. It'd be
really nice if we could have a little little discussion just to see what the
plan is like and what sort of features are going to expose to userspace. This is
all a very rapid development which is good. I'd just like to make sure to know
what we're in for. Sort of like what we did last year at Plumber but less
offical and without microphones and the overhead of having hundreds of people in
the room. :) Also it likely be way more relaxed given that a lot of the problems
dicussed back then seem to have been resolved.

CCing Serge and Stéphane. :)

Thanks!
Christian

> 
> Sure, the only thing I think we need for cpuset is cutting down the
> interface to the minimal set which provides all the features to
> userspace.
> 
> Thanks.
> 
> -- 
> tejun
> --
> To unsubscribe from this list: send the line "unsubscribe cgroups" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Christian Brauner]

On Thu, Aug 24, 2017 at 08:02:22AM -0700, Tejun Heo wrote:
> Hello,
> 
> On Thu, Aug 24, 2017 at 10:56:21AM -0400, Waiman Long wrote:
> > The main reason for sending out this patch is to figure out what exactly
> > is in your mind before enabling cpuset in v2.
> > 
> > From my point of view, the exclusive setting makes cpuset behave more
> > like a resource domain that a resource allocated to one cgroup won't be
> > available for another cgroup. So we can argue it both ways whether it
> > violates the basic hierarchy rules or not.
> 
> It's not even siblings interfering with other siblings.  The resource
> knobs belong to the parent, right?  So, the only thing it's doing is
> restricting the parent itself from creating certain configurations,
> which can be argued to be a feature but it's a really weird policy
> enforcement implemented in kernel.
> 
> > I will be in the Plumbers Conference next month and we can talk more
> > about this. My goal is to make cgroup v2 ready for prime time hopefully
> > by the end of the year.

Hi,

I've been following this thread and the cpuset controller being enabled on
cgroup v2 is a really important step. As one of many container runtime
maintainers this is obviously a big step for us with a lot of impact. It'd be
really nice if we could have a little little discussion just to see what the
plan is like and what sort of features are going to expose to userspace. This is
all a very rapid development which is good. I'd just like to make sure to know
what we're in for. Sort of like what we did last year at Plumber but less
offical and without microphones and the overhead of having hundreds of people in
the room. :) Also it likely be way more relaxed given that a lot of the problems
dicussed back then seem to have been resolved.

CCing Serge and Stéphane. :)

Thanks!
Christian

> 
> Sure, the only thing I think we need for cpuset is cutting down the
> interface to the minimal set which provides all the features to
> userspace.
> 
> Thanks.
> 
> -- 
> tejun
> --
> To unsubscribe from this list: send the line "unsubscribe cgroups" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Mike Galbraith]

On Thu, 2017-08-24 at 07:43 -0700, Tejun Heo wrote:
> Hello, Waiman.
> 
> On Thu, Aug 24, 2017 at 10:40:11AM -0400, Waiman Long wrote:
> > Given the fact that thread mode is now queued for 4.14, it is now
> > time to enable cpuset to be used in the default hierarchy (cgroup v2)
> > as it is clearly threaded.
> 
> I'm not sure we want things like exclusive settings in v2 as it
> directly violates basic hierarchy rules (a sibling shouldn't be able
> to affect what another sibling can or can't do).  We need to think
> more about what should be exposed in v2.

I was wondering when this was gonna come up.  The cpu/cpuacct stuff
seems well on its way to resolution, but partitioning (critical) is sti
ll dangling in the breeze.

	-Mike

Re: [PATCH] cpuset: Allow cpuset controller in default hierarchy

[Mike Galbraith]

On Thu, 2017-08-24 at 07:43 -0700, Tejun Heo wrote:
> Hello, Waiman.
> 
> On Thu, Aug 24, 2017 at 10:40:11AM -0400, Waiman Long wrote:
> > Given the fact that thread mode is now queued for 4.14, it is now
> > time to enable cpuset to be used in the default hierarchy (cgroup v2)
> > as it is clearly threaded.
> 
> I'm not sure we want things like exclusive settings in v2 as it
> directly violates basic hierarchy rules (a sibling shouldn't be able
> to affect what another sibling can or can't do).  We need to think
> more about what should be exposed in v2.

I was wondering when this was gonna come up.  The cpu/cpuacct stuff
seems well on its way to resolution, but partitioning (critical) is sti
ll dangling in the breeze.

	-Mike