From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965129AbWFASoD (ORCPT ); Thu, 1 Jun 2006 14:44:03 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S965212AbWFASoC (ORCPT ); Thu, 1 Jun 2006 14:44:02 -0400 Received: from e34.co.us.ibm.com ([32.97.110.152]:33497 "EHLO e34.co.us.ibm.com") by vger.kernel.org with ESMTP id S965129AbWFASoA (ORCPT ); Thu, 1 Jun 2006 14:44:00 -0400 Subject: Re: [ckrm-tech] [RFC 3/5] sched: Add CPU rate hard caps From: Chandra Seetharaman Reply-To: sekharan@us.ibm.com To: balbir@in.ibm.com, dev@openvz.org Cc: Andrew Morton , Srivatsa , Sam Vilain , ckrm-tech@lists.sourceforge.net, Balbir Singh , Mike Galbraith , Peter Williams , Con Kolivas , Linux Kernel , Kingsley Cheung , "Eric W. Biederman" , Ingo Molnar , Rene Herman In-Reply-To: <447EA694.8060407@in.ibm.com> References: <20060526042021.2886.4957.sendpatchset@heathwren.pw.nest> <20060526042051.2886.70594.sendpatchset@heathwren.pw.nest> <661de9470605262348s52401792x213f7143d16bada3@mail.gmail.com> <44781167.6060700@bigpond.net.au> <447D95DE.1080903@sw.ru> <447DBD44.5040602@in.ibm.com> <447E9A1D.9040109@openvz.org> <447EA694.8060407@in.ibm.com> Content-Type: text/plain Organization: IBM Date: Thu, 01 Jun 2006 11:43:33 -0700 Message-Id: <1149187413.13336.24.camel@linuxchandra> Mime-Version: 1.0 X-Mailer: Evolution 2.0.4 (2.0.4-7) Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2006-06-01 at 14:04 +0530, Balbir Singh wrote: > Hi, Kirill, > > Kirill Korotaev wrote: > >> Do you have any documented requirements for container resource > >> management? > >> Is there a minimum list of features and nice to have features for > >> containers > >> as far as resource management is concerned? > > > > Sure! You can check OpenVZ project (http://openvz.org) for example of > > required resource management. BTW, I must agree with other people here > > who noticed that per-process resource management is really useless and > > hard to use :( > I totally agree. > I'll take a look at the references. I agree with you that it will be useful > to have resource management for a group of tasks. > > > > > Briefly about required resource management: > > 1) CPU: > > - fairness (i.e. prioritization of containers). For this we use SFQ like > > fair cpu scheduler with virtual cpus (runqueues). Linux-vserver uses > > tocken bucket algorithm. I can provide more details on this if you are > > interested. > > Yes, any information or pointers to them will be very useful. > > > - cpu limits (soft, hard). OpenVZ provides only hard cpu limits. For > > this we account the time in cycles. And after some credit is used do > > delay of container execution. We use cycles as our experiments show that > > statistical algorithms work poorly on some patterns :( > > - cpu guarantees. I'm not sure any of solutions provide this yet. > > ckrm has a solution to provide cpu guarantees. > > I think as far as CPU resource management is concerned (limits or guarantees), > there are common problems to be solved, for example > > 1. Tracking when a limit or a gaurantee is not met > 2. Taking a decision to cap the group > 3. Selecting the next task to execute (keeping O(1) in mind) > > For the existing resource controller in OpenVZ I would be > interested in the information on the kinds of patterns it does not > perform well on and the patterns it performs well on. > > > > > 2) disk: > > - overall disk quota for container > > - per-user/group quotas inside container > > > > in OpenVZ we wrote a 2level disk quota which works on disk subtrees. > > vserver imho uses 1 partition per container approach. > > > > - disk I/O bandwidth: > > we started to use CFQv2, but it is quite poor in this regard. First, it > > doesn't prioritizes writes and async disk operations :( And even for > > sync reads we found some problems we work on now... CKRM (on e-series) had an implementation based on a modified CFQ scheduler. Shailabh is currently working on porting that controller to f-series. > > > > 3) memory and other resources. > > - memory > > - files > > - signals and so on and so on. > > For example, in OpenVZ we have user resource beancounters (original > > author is Alan Cox), which account the following set of parameters: > > kernel memory (vmas, page tables, different structures etc.), dcache i started looking at UBC. They provide only max limits, not min guarantees, right ? > > pinned size, different user pages (locked, physical, private, shared), > > number of files, sockets, ptys, signals, network buffers, netfilter > > rules etc. > > > -- ---------------------------------------------------------------------- Chandra Seetharaman | Be careful what you choose.... - sekharan@us.ibm.com | .......you may get it. ----------------------------------------------------------------------