Re: Single process controlling all cgroups sounds like looking in right direction but wrong solution.

Re: Single process controlling all cgroups sounds like looking in right direction but wrong solution.

[Peter Dolding]

I followed the Maintainers File.  https://www.kernel.org/doc/linux/MAINTAINERS
CONTROL GROUPS (CGROUPS)
M:	Paul Menage <menage-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
M:	Li Zefan <lizf-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
L:	containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
S:	Maintained
F:	include/linux/cgroup*
F:	kernel/cgroup*
F:	mm/*cgroup*


Apparently by your response this might be a bit out of date.  I just read
lwm and *Tejun Heo is not even as a main maintainer.  Listed as a sub part
maintainer.   By the maintainers file discussions should be in *
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org where I sent this.

Tejun Heo please inform if this is still correct.  Its either update this
or tell lwn.net to get your title correct in future.

Serge I am trying to follow policy that is why I posted here in the first
place.

On Mon, Jul 15, 2013 at 1:25 PM, Serge Hallyn <serge.hallyn-GeWIH/nMZzLQT0dZR+AlfA@public.gmane.org>wrote:

> Hi Peter,
>
> Interesting topic.
>
> I think this issue is best discussed on the cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> mailing list.
>
> -serge
>
> Quoting Peter Dolding (oiaohm-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org):
> > NUMA is where the fun starts.   If I wish to adjust a group of processes
> > bound only to a NUMA group of cpus.   It should not be required to
> disrupt
> > other cpus.
> >
> > Init system controlling the cgroups is also sounding trouble.   Why we
> > don't want to write like 1 000 000 different interfaces to talk to
> cgroups.
> >
> > Items like chromuim browser might wish to use cgroups in future to lets
> say
> > contain flash.
> >
> > Best solution to the problems  not a library or in the init system its a
> > form of userspace driver.
> >
> > The userspace driver has the permissions to alter and change cgroups and
> > possible no right todo anything else in final form.
> >
> > Why not part of the init binary.   Lets say you have 1000 processors.
> > And for some reason you are running something that tweaking cgroups a
> > lot.   You don't want to stall all 1000 processes if it not required.
> > Single process could cause this.   What ever in in change of the cgroups
> > has to be massively multi threaded.    Also has to be changeable based on
> > NUMA requirements of the system at times.   You don't want to have to be
> > editing the core of init system or reboot the system just to change the
> > cgroup management system to be more compatible with current hardware.
>  This
> > is again why userspace driver.  You can stop the driver and start another
> > one if required.   While still maintaining the rule of only 1 active at
> any
> > 1 time.  If someone can load a kernel driver any how the can cause hell.
> >
> > Single process is not going to work because this will require stopping
> all
> > cpus.   What is required is single processes/theads.
> >
> > Going the userspace driver solution emulation of the old cgroup system
> > could be pushed into userspace.   So removing old code from the core and
> > allow rejecting of hazard changes to be applied to the old method without
> > having to update kernel every time.
> >
> > If there is an requirement to change the interface in future its less
> > problem.
> >
> > If kbus (kernel dbus) was already done I would be tempted to place this
> as
> > a default feature on the kbus.   This would not be dependant on any
> > particular init system.
> >
> > Since kbus is not ready creating a text device to receive messages for
> now
> > would be a suitable solution in my eyes.
> >
> > Peter Dolding
> > _______________________________________________
> > Containers mailing list
> > Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> > https://lists.linuxfoundation.org/mailman/listinfo/containers
>

Re: Single process controlling all cgroups sounds like looking in right direction but wrong solution.

[Peter Dolding]

Sorry everyone I forgot Plain text this from google.   vger and its
hate of html.


On Mon, Jul 15, 2013 at 1:47 PM, Peter Dolding <oiaohm-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> I followed the Maintainers File.  https://www.kernel.org/doc/linux/MAINTAINERS
> CONTROL GROUPS (CGROUPS)
> M: Paul Menage <menage-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
> M: Li Zefan <lizf-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
> L: containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> S: Maintained
> F: include/linux/cgroup*
> F: kernel/cgroup*
> F: mm/*cgroup*
>
>
> Apparently by your response this might be a bit out of date.  I just read lwm and Tejun Heo is not even as a main maintainer.  Listed as a sub part maintainer.   By the maintainers file discussions should be in containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org where I sent this.
>
> Tejun Heo please inform if this is still correct.  Its either update this or tell lwn.net to get your title correct in future.
>
> Serge I am trying to follow policy that is why I posted here in the first place.
>
> On Mon, Jul 15, 2013 at 1:25 PM, Serge Hallyn <serge.hallyn-GeWIH/nMZzLQT0dZR+AlfA@public.gmane.org> wrote:
>>
>> Hi Peter,
>>
>> Interesting topic.
>>
>> I think this issue is best discussed on the cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> mailing list.
>>
>> -serge
>>
>> Quoting Peter Dolding (oiaohm-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org):
>> > NUMA is where the fun starts.   If I wish to adjust a group of processes
>> > bound only to a NUMA group of cpus.   It should not be required to disrupt
>> > other cpus.
>> >
>> > Init system controlling the cgroups is also sounding trouble.   Why we
>> > don't want to write like 1 000 000 different interfaces to talk to cgroups.
>> >
>> > Items like chromuim browser might wish to use cgroups in future to lets say
>> > contain flash.
>> >
>> > Best solution to the problems  not a library or in the init system its a
>> > form of userspace driver.
>> >
>> > The userspace driver has the permissions to alter and change cgroups and
>> > possible no right todo anything else in final form.
>> >
>> > Why not part of the init binary.   Lets say you have 1000 processors.
>> > And for some reason you are running something that tweaking cgroups a
>> > lot.   You don't want to stall all 1000 processes if it not required.
>> > Single process could cause this.   What ever in in change of the cgroups
>> > has to be massively multi threaded.    Also has to be changeable based on
>> > NUMA requirements of the system at times.   You don't want to have to be
>> > editing the core of init system or reboot the system just to change the
>> > cgroup management system to be more compatible with current hardware.  This
>> > is again why userspace driver.  You can stop the driver and start another
>> > one if required.   While still maintaining the rule of only 1 active at any
>> > 1 time.  If someone can load a kernel driver any how the can cause hell.
>> >
>> > Single process is not going to work because this will require stopping all
>> > cpus.   What is required is single processes/theads.
>> >
>> > Going the userspace driver solution emulation of the old cgroup system
>> > could be pushed into userspace.   So removing old code from the core and
>> > allow rejecting of hazard changes to be applied to the old method without
>> > having to update kernel every time.
>> >
>> > If there is an requirement to change the interface in future its less
>> > problem.
>> >
>> > If kbus (kernel dbus) was already done I would be tempted to place this as
>> > a default feature on the kbus.   This would not be dependant on any
>> > particular init system.
>> >
>> > Since kbus is not ready creating a text device to receive messages for now
>> > would be a suitable solution in my eyes.
>> >
>> > Peter Dolding
>> > _______________________________________________
>> > Containers mailing list
>> > Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
>> > https://lists.linuxfoundation.org/mailman/listinfo/containers
>
>

Re: Single process controlling all cgroups sounds like looking in right direction but wrong solution.

[Serge Hallyn]

Quoting Peter Dolding (oiaohm-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org):
> I followed the Maintainers File.  https://www.kernel.org/doc/linux/MAINTAINERS
> CONTROL GROUPS (CGROUPS)
> M:	Paul Menage <menage-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
> M:	Li Zefan <lizf-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
> L:	containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org

Odd, my version has

L:      containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
L:      cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org

The cgroups entry was added in November 2011 according to git-blame.
I don't know why the kernel.org version is so old.

Still I think that should be patched to remove containers@.  I
originally objected to the cgroup@ list creation, but since I
do not believe the relevant cgroup folks read the containers@
list any more, I don't think containers@ should be listed -
certainly not first.

> S:	Maintained
> F:	include/linux/cgroup*
> F:	kernel/cgroup*
> F:	mm/*cgroup*
> 
> 
> Apparently by your response this might be a bit out of date.  I just read
> lwm and *Tejun Heo is not even as a main maintainer.  Listed as a sub part
> maintainer.   By the maintainers file discussions should be in *
> containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org where I sent this.
> 
> Tejun Heo please inform if this is still correct.  Its either update this
> or tell lwn.net to get your title correct in future.
> 
> Serge I am trying to follow policy that is why I posted here in the first
> place.

That sounds unnecessarily defensive - I wasn't complaining, just trying
to help your email get to where it would be best discussed  :)  Sorry
that it involves an extra step (resending), but I didn't want to simply
reply cc:ing cgroups@, as the email thread tends to get funky that way.

-serge

Re: Single process controlling all cgroups sounds like looking in right direction but wrong solution.

[Peter Dolding]

Serge You took it the wrong way I was not trying to be defensive.
That was my version of what the heck is wrong response.  Followed
policy and it wrong you kinda want to find out what is wrong.  Its if
something is out like this other people are going to go south.

So this is interesting there is a miss match between git and what the
website displays.   Fact that its old it the webmaster needs to be
informed so it can be fixed.      I have changed the who this is going
to.

Webmaster https://www.kernel.org/doc/linux/MAINTAINERS   and the
Maintainers file in git don't match.    Would suspect other sections
in doc directory are also out.

https://www.kernel.org/doc/linux/   Apparently this is one of the
things that broke when kernel.org was breached and has not been fixed
yet.   Nothing in that section of kernel.org appears to be updated
since the 19 Aug 2011.   Kernel.org breach was the 31 of Aug 2011.
So I would say the script or whatever was done to sync there has not
been operating because it was removed cleaning up from the breach.
If its never going to be operating sections there should be just
deleted so it does not lead people up garden path..

On Mon, Jul 15, 2013 at 10:32 PM, Serge Hallyn <serge.hallyn-GeWIH/nMZzLQT0dZR+AlfA@public.gmane.org> wrote:
> Quoting Peter Dolding (oiaohm-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org):
>> I followed the Maintainers File.  https://www.kernel.org/doc/linux/MAINTAINERS
>> CONTROL GROUPS (CGROUPS)
>> M:    Paul Menage <menage-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
>> M:    Li Zefan <lizf-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
>> L:    containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
>
> Odd, my version has
>
> L:      containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> L:      cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>
> The cgroups entry was added in November 2011 according to git-blame.
> I don't know why the kernel.org version is so old.
>
> Still I think that should be patched to remove containers@.  I
> originally objected to the cgroup@ list creation, but since I
> do not believe the relevant cgroup folks read the containers@
> list any more, I don't think containers@ should be listed -
> certainly not first.
>
>> S:    Maintained
>> F:    include/linux/cgroup*
>> F:    kernel/cgroup*
>> F:    mm/*cgroup*
>>
>>
>> Apparently by your response this might be a bit out of date.  I just read
>> lwm and *Tejun Heo is not even as a main maintainer.  Listed as a sub part
>> maintainer.   By the maintainers file discussions should be in *
>> containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org where I sent this.
>>
>> Tejun Heo please inform if this is still correct.  Its either update this
>> or tell lwn.net to get your title correct in future.
>>
>> Serge I am trying to follow policy that is why I posted here in the first
>> place.
>
> That sounds unnecessarily defensive - I wasn't complaining, just trying
> to help your email get to where it would be best discussed  :)  Sorry
> that it involves an extra step (resending), but I didn't want to simply
> reply cc:ing cgroups@, as the email thread tends to get funky that way.
>
> -serge

Re: Single process controlling all cgroups sounds like looking in right direction but wrong solution.

[Konstantin Ryabitsev]

[-- Attachment #1: Type: text/plain, Size: 746 bytes --]

On 16/07/13 06:06 PM, Peter Dolding wrote:
> Webmaster https://www.kernel.org/doc/linux/MAINTAINERS   and the
> Maintainers file in git don't match.    Would suspect other sections
> in doc directory are also out.

Correct, that seems to have been completely overlooked. I just put
together a script that will auto-build and update the documentation
tree. The following is now auto-updated each time Linus tags his tree:

https://www.kernel.org/doc/Documentation/
https://www.kernel.org/doc/htmldocs/
https://www.kernel.org/doc/readme/
https://www.kernel.org/doc/linux/
https://www.kernel.org/doc/makehelp.txt

Best,
-- 
Konstantin Ryabitsev
Senior Systems Administrator
Linux Foundation Collab Projects
Montréal, Québec


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 730 bytes --]

Re: Single process controlling all cgroups sounds like looking in right direction but wrong solution.

[Rob Landley]

On 07/15/2013 07:32:16 AM, Serge Hallyn wrote:
> Quoting Peter Dolding (oiaohm-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org):
> > I followed the Maintainers File.   
> https://www.kernel.org/doc/linux/MAINTAINERS
> > CONTROL GROUPS (CGROUPS)
> > M:	Paul Menage <menage-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
> > M:	Li Zefan <lizf-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
> > L:	containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> 
> Odd, my version has
> 
> L:      containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> L:      cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> 
> The cgroups entry was added in November 2011 according to git-blame.
> I don't know why the kernel.org version is so old.

Because when kernel.org got broken into they did a major  
barn-door-locking that took away everyone's account until you could get  
your key signed by senior kernel developers in person, and since I  
don't go to a lot of conferences I didn't manage that until February of  
this year.

Then when I got my account back, I found out that the ability to rsync  
over ssh had gone away and instead they've replaced shell access with a  
home-grown tool called "kup" (because as well all know, the way to  
secure a system is for non-security people to write their own tools  
from scratch). And unfortunately, that tool is basically "git access  
and afterthoughts".

It's theoretically possible to copy files through kup, one at a time,  
after individually cryptographically signing each file. It's also  
possible to list directories through kup. So what I need to do is write  
a shell script that traverses my local kdocs directory, lists the  
contents on the website, makes two trees, compares the trees, figures  
out which files need updating, signs each file and uploads it.

I.E. laboriously reimplementing a sad immitation of rsync through this  
insane bespoke tool.

It's on my todo list...

Rob

(P.S. yes I asked: the kernel developers do not care in the slightest  
that when the server changed out from under the users, their new tool  
does not match my existing workflow. And it's their server, they can do  
what they like. I'm just annoyed at the "department of homeland  
security" levels of disproportionate response, and that containers are  
still not considered worth using here.)--
To unsubscribe from this list: send the line "unsubscribe cgroups" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html