Re: [Libcg-devel] [RFC] How to handle the rules engine for cgroups

[Dhaval Giani <dhaval@linux.vnet.ibm.com>]

On Thu, Jul 17, 2008 at 10:37:17PM +0530, Sudhir Kumar wrote:
> On Thu, Jul 17, 2008 at 09:47:36AM -0400, Vivek Goyal wrote:
> > On Thu, Jul 17, 2008 at 04:05:17PM +0900, Kazunaga Ikeno wrote:
> > > Vivek Goyal wrote:
> > > > On Mon, Jul 14, 2008 at 10:44:43AM -0400, David Collier-Brown wrote:
> > > > > Vivek Goyal wrote:
> > > > >> If admin has decided to group applications and has written the rules for
> > > > >> it then applications should not know anything about grouping. So I think
> > > > >> application writing an script for being placed into the right group should
> > > > >> be out of question. Now how does an admin write a wrapper around existing
> > > > >> application without breaking anything else.
> > > > >
> > > > > In the Solaris world, processes are placed into cgroups (projects) by
> > > > > one of two mechanisms:
> > > > >
> > > > > 1) inheritance, with everything I create in my existing project.
> > > > >   To get this started, there is a mechanism under login/getty/whatever
> > > > > reading the /etc/projects file and, for example, tossing user davecb
> > > > > into a "user.davecb" project.
> > > > >
> > > > 
> > > > Placing the login sessions in right cgroup based on uid/gid rules is
> > > > probably easy as check needs to be placed only on system entry upon login
> > > > (Pam plugin should do).  And after that any job started by the user
> > > > will automatically start in the same cgroup.
> > > > 
> > > > > 2) explicit placement with newtask, which starts a program or moves
> > > > >   a process into a project/cgroup
> > > > >
> > > > 
> > > > explicit placement of task based on application type will be tricky.
> > > > 
> > > > > I have a "bg" project which I use for limiting resource consumption of
> > > > > background jobs, and a background command which either starts or moves
> > > > > jobs, thusly:
> > > > >
> > > > > case "$1" in
> > > > > [0-9]*) # It's a pid
> > > > >         newtask -p bg -c $1
> > > > 
> > > > Ok, this is moving of tasks from one cgroup to other based on pid. This
> > > > is really easy to do through cgroup file system. Just a matter of writing
> > > > to task file.
> > > > 
> > > > >         ;;
> > > > >  *) # It's a command-line
> > > > >         newtask -p bg "$@" &
> > > > >         ;;
> > > > 
> > > > So here a user explicitly invokes the wrapper passing it the targeted
> > > > cgroup and the application to be launched in that cgroup. This should work
> > > > if there is a facility if user has created its own cgroups (lets say
> > > > under user controlled cgroup dir in the hierarchy) and user explicitly
> > > > wants to control the resources of applications under its dir. For example,
> > > > 
> > > >  		/mnt/cgroup
> > > > 		|	|
> > > > 		gid1	gid2
> > > > 		|  |	|  |
> > > > 	      uid1 uid2	uid3 uid4
> > > > 	     |  |
> > > > 	 proj1  proj2
> > > > 
> > > > Here probably admin can write the rules for how users are allocated the
> > > > resources and give ability to users to create subdirs under their cgroups
> > > > where users can create more cgroups and can do their own resource
> > > > management based on application tasks and place applications in the right
> > > > cgroup by writing wrappers as mentioned by you "newtask".
> > > > 
> > > > But here there is no discrimination of application type by admin. Admin
> > > > controls resource divisions only based on uid/gid. And users can manage
> > > > applications within their user groups. In fact I am having hard time thinking
> > > > in what kind of scenarios, there is a need for an admin to control
> > > > resource based on application type? Do we really need setups like, on
> > > > a system databases should get network bandwidth of 30%. If yes, then
> > > > it becomes tricky where admin need to write a wrapper to place the task
> > > > in right cgroup without application/user knowing it.
> > > 
> > > I think a wrapper (move to right group and calls exec) will run by user, not by admin.
> > > In explicit placement, user knows what a type of application he/she launch.
> > > 
> > >  		/mnt/cgroup
> > > 		|	|
> > > 		gid1	gid2
> > > 		|  |	|  |
> > > 	      uid1 uid2	uid3 uid4
> > > 	     |  |
> > > 	 proj1  proj2
> > > 
> > 
> > This is the easy to handle situation and I am hoping it will work in many
> > of the cases.
> 
> This solution seems ok but this looks only one part of the storey. Here
> the top level hierarchy is again user based(gid/uid). What if admin
> wants to manage the system resources per application basis? Say a big
> server in a university is being shared by everyone in the university for
> only 3 application
> 1. http server
> 2. browsing
> 3. computing
> In case the admin wants the system to be always available for computing,
> how should he utilize cgroups for managing the server resources among
> these applications ?
> Isn't such scenarios on the priority now?

We only have FS permissions to play around with. Therefore any hierarchy
we come up with will be uid/gid based. Such scenarios will be handled by
the administrator by ensuring the correct permissions are set for the
cgroup.

> 
> > 
> > Currently I am writting a patch for libcg which allows querying the
> > destination cgroup based on uid/gid and libcg will also migrate the
> > application there. I am also writing a pam plugin which will move
> > all the login sessions to respective cgroup (as mentioned by rule file).
> > Will also modify "init" so that all the system services to into cgroup
> > belonging to root.
> > 
> > Once user is logged in and running into his resource group, he can manage
> > further subgroups at his own based on his application requirements (as you
> > mentioned proj1 and proj2 here).
> > 
> > > [uid1/gid1]% newtask.sh proj1app
> > > ... proj1app run under /mnt/cgroup/gid1/uid1
> > > 
> > 
> > Yes, so if a user does not specifically launch an application targetted
> > for a particular cgroup, then it will run into default group for that
> > user (as specified by rule file). In this case under /mnt/cgroup/gid1/uid1.
> So in this user based approach if admin wants to run 4 major
> applications each one requiring say 15% cpu he needs to create 4
> different gids? Creation of a user account just for running an
> application does not look very flexible to me.
> 

A lot of daemons run as specific users. Also its not a good idea to run
daemons/servers as root users. They should run as users who have limited
privileges. With such a model in place, Vivek's comments make sense and
might be the right way to go ahead. 

Thanks,
-- 
regards,
Dhaval