Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

[Michael Kerrisk (man-pages)]

Hello Tejun,

Suppose we have the following scenario:

x [d]    (pids)
    y [dt]    (pids)
        p [t]
        q [t]
        r [t]
    z [d]

Here, x/y is a "domain threaded root" with a threaded controller
(the 'pids' controller) enabled. (In this scenario, there are no
member processes in any of the cgroups.)

Suppose we now convert x/z to "threaded" type:

     # echo threaded > x/z/cgroup.type

Now we end up in the following state:

x [dt]    (pids)
    y [inv]    (pids)
        p [t]
        q [t]
        r [t]
    z [t]

This seems odd. x/y is now of "domain invalid" type with a controller
enabled! This feels like a violation of the rules, since we can't
in other circumstances do anything with a "domain invalid" cgroup
except convert it to "threaded". In particular, we can't create
child cgroups under a "domain invalid" cgroup, or add member processes
to the cgroup, or *enable controllers in the cgroup*. In fact, when
doing the 

    # echo threaded > x/z/cgroup.type

I had expected a write(2) error because the state of x/y should
(I thought) not be permitted.

Your thoughts?

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

[Roman Gushchin]

On Thu, Oct 04, 2018 at 09:40:57PM +0200, Michael Kerrisk (man-pages) wrote:
> Hello Tejun,
> 
> Suppose we have the following scenario:
> 
> x [d]    (pids)
>     y [dt]    (pids)
>         p [t]
>         q [t]
>         r [t]
>     z [d]
> 
> Here, x/y is a "domain threaded root" with a threaded controller
> (the 'pids' controller) enabled. (In this scenario, there are no
> member processes in any of the cgroups.)
> 
> Suppose we now convert x/z to "threaded" type:
> 
>      # echo threaded > x/z/cgroup.type
> 
> Now we end up in the following state:
> 
> x [dt]    (pids)
>     y [inv]    (pids)
>         p [t]
>         q [t]
>         r [t]
>     z [t]
> 
> This seems odd. x/y is now of "domain invalid" type with a controller
> enabled! This feels like a violation of the rules, since we can't
> in other circumstances do anything with a "domain invalid" cgroup
> except convert it to "threaded". In particular, we can't create
> child cgroups under a "domain invalid" cgroup, or add member processes
> to the cgroup, or *enable controllers in the cgroup*. In fact, when
> doing the 
> 
>     # echo threaded > x/z/cgroup.type
> 
> I had expected a write(2) error because the state of x/y should
> (I thought) not be permitted.
> 
> Your thoughts?

Just a note: there are now some cgroup core kernel selftests in
tools/testing/selftests/cgroup/test_core.c . It's fairly easy to add new
tests to cover tricky cases like this one. It always nice to have more tests,
and also easier to show and reproduce a problem.

Thanks!

Re: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

[Michael Kerrisk (man-pages)]

Hello Tejun,

A ping on the below...

Thanks,

Michael

On 10/04/2018 09:40 PM, Michael Kerrisk (man-pages) wrote:
> Hello Tejun,
> 
> Suppose we have the following scenario:
> 
> x [d]    (pids)
>     y [dt]    (pids)
>         p [t]
>         q [t]
>         r [t]
>     z [d]
> 
> Here, x/y is a "domain threaded root" with a threaded controller
> (the 'pids' controller) enabled. (In this scenario, there are no
> member processes in any of the cgroups.)
> 
> Suppose we now convert x/z to "threaded" type:
> 
>      # echo threaded > x/z/cgroup.type
> 
> Now we end up in the following state:
> 
> x [dt]    (pids)
>     y [inv]    (pids)
>         p [t]
>         q [t]
>         r [t]
>     z [t]
> 
> This seems odd. x/y is now of "domain invalid" type with a controller
> enabled! This feels like a violation of the rules, since we can't
> in other circumstances do anything with a "domain invalid" cgroup
> except convert it to "threaded". In particular, we can't create
> child cgroups under a "domain invalid" cgroup, or add member processes
> to the cgroup, or *enable controllers in the cgroup*. In fact, when
> doing the 
> 
>     # echo threaded > x/z/cgroup.type
> 
> I had expected a write(2) error because the state of x/y should
> (I thought) not be permitted.
> 
> Your thoughts?
> 
> Thanks,
> 
> Michael
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

[Tejun Heo]

Hello, Michael.

Sorry about the delay.

On Thu, Oct 04, 2018 at 09:40:57PM +0200, Michael Kerrisk (man-pages) wrote:
> This seems odd. x/y is now of "domain invalid" type with a controller
> enabled! This feels like a violation of the rules, since we can't
> in other circumstances do anything with a "domain invalid" cgroup
> except convert it to "threaded". In particular, we can't create
> child cgroups under a "domain invalid" cgroup, or add member processes
> to the cgroup, or *enable controllers in the cgroup*. In fact, when
> doing the 
> 
>     # echo threaded > x/z/cgroup.type
> 
> I had expected a write(2) error because the state of x/y should
> (I thought) not be permitted.

So, both the interim (before turning x/z into threaded) and final
(after) are completely fine - the cgroups are empty and whether
threaded controllers like pids are enabled or not don't really change
things that much.

Maybe it is a bit inconsistent to then deny enabling threaded
controllers on invalid domain cgroups.  We can lift that restriction
but I personally can't see why that'd be clearly better.

Thanks.

-- 
tejun

Re: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

[Michael Kerrisk (man-pages)]

Hi Tejun
On Wed, 17 Oct 2018 at 18:20, Tejun Heo <tj@kernel.org> wrote:
>
> Hello, Michael.
>
> Sorry about the delay.
>
> On Thu, Oct 04, 2018 at 09:40:57PM +0200, Michael Kerrisk (man-pages) wrote:
> > This seems odd. x/y is now of "domain invalid" type with a controller
> > enabled! This feels like a violation of the rules, since we can't
> > in other circumstances do anything with a "domain invalid" cgroup
> > except convert it to "threaded". In particular, we can't create
> > child cgroups under a "domain invalid" cgroup, or add member processes
> > to the cgroup, or *enable controllers in the cgroup*. In fact, when
> > doing the
> >
> >     # echo threaded > x/z/cgroup.type
> >
> > I had expected a write(2) error because the state of x/y should
> > (I thought) not be permitted.
>
> So, both the interim (before turning x/z into threaded) and final
> (after) are completely fine - the cgroups are empty and whether
> threaded controllers like pids are enabled or not don't really change
> things that much.
>
> Maybe it is a bit inconsistent to then deny enabling threaded
> controllers on invalid domain cgroups.  We can lift that restriction
> but I personally can't see why that'd be clearly better.

So, I also can't see anything that seems harmful in the scenario I
described; it's just an odd inconsistency, and I supposed it was
unintended/overlooked behavior, and I wanted to alert you to it, in
case it seemed problematic. However, as long as you have no concerns,
I see no reason to change the existing behavior.

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/