All of lore.kernel.org
 help / color / mirror / Atom feed
From: Pavel Emelyanov <xemul-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
To: Christoph Lameter <clameter-sJ/iWh9BUns@public.gmane.org>
Cc: Linux Containers
	<containers-qjLDD68F18O7TbgM5vRIOg@public.gmane.org>,
	Balbir Singh
	<balbir-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Subject: [PATCH 0/5] Kernel memory accounting container (v5)
Date: Tue, 25 Sep 2007 18:16:33 +0400	[thread overview]
Message-ID: <46F91841.9070708@openvz.org> (raw)

Changes since v.4:
* make SLAB_NOTIFY caches mark pages as SlabDebug. That
  makes the interesting paths simpler (thanks to Christoph);
* the change above caused appropriate changes in "turn
  notifications on" path - all available pages must become 
  SlabDebug and page's freelists must be flushed;
* added two more events - "on" and "off" to make kmalloc
  caches disabling more gracefully;
* turning notifications "off" is marked as "TODO". Right
  now it's hard w/o massive rework of slub.c in respect to
  full slabs handling.

Changes since v.3:
* moved alloc/free notification into slow path and make
  "notify-able" caches walk this path always;
* introduced some optimization for the case, when there's
  only one listener for SLUB events (saves more that 10%
  of performance);
* ported on 2.6.23-rc6-mm1 tree.

Changes since v.2:
* introduced generic notifiers for slub. right now there
  are only events, needed by accounting, but this set can
  be extended in the future;
* moved the controller core into separate file, so that
  its extension and/or porting on slAb will look more 
  logical;
* fixed this message :).

Changes since v.1:
* fixed Paul's comment about subsystem registration;
* return ERR_PTR from ->create callback, not NULL;
* make container-to-object assignment in rcu-safe section;
* make turning accounting on and off with "1" and "0".

============================================================

Long time ago we decided to start memory control with the
user memory container. Now this container in -mm tree and
I think we can start with the kmem one.

First of all - why do we need this kind of control. The major
"pros" is that kernel memory control protects the system
from DoS attacks by processes that live in container. As our
experience shows many exploits simply do not work in the
container with limited kernel memory.

I can split the kernel memory container into 4 parts:

1. kmalloc-ed objects control
2. vmalloc-ed objects control
3. buddy allocated pages control
4. kmem_cache_alloc-ed objects control

the control of first tree types of objects has one peculiarity:
one need to explicitly point out which allocations he wants to
account and this becomes not-configurable and is to be discussed.

On the other hands such objects as anon_vma-s, file-s, sighangds,
vfsmounts, etc are created by user request always and should
always be accounted. Fortunately they are allocated from their
own caches and thus the whole kmem cache can be accountable.

This is exactly what this patchset does - it adds the ability
to account for the total size of kmem-cache-allocated objects
from specified kmem caches.

This is based on the SLUB allocator, Paul's control groups and the
resource counters I made for RSS controller and which are in
-mm tree already.

To play with it, one need to mount the container file system
with -o kmem and then mark some caches as accountable via
/sys/slab/<cache_name>/cache_notify.

As I have already told kmalloc caches cannot be accounted easily
so turning the accounting on for them will fail with -EINVAL.

Turning the accounting off is possible only if the cache has
no objects. This is done so because turning accounting off
implies marking of all the slabs in the cache as not-debug, but 
due to full-pages in slub are not stored in any lists (usually)
this is impossible to do so, however this is in todo list.

Thanks,
Pavel

             reply	other threads:[~2007-09-25 14:16 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-09-25 14:16 Pavel Emelyanov [this message]
     [not found] ` <46F91841.9070708-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-09-25 14:18   ` [PATCH 1/5] Add notification about some major slab events Pavel Emelyanov
     [not found]     ` <46F91898.5060400-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-09-25 21:47       ` Christoph Lameter
     [not found]         ` <Pine.LNX.4.64.0709251445400.5072-RYO/mD75kfhx2SFC9UQUAuF7EQX82lMiAL8bYrjMMd8@public.gmane.org>
2007-09-26  9:37           ` Pavel Emelyanov
     [not found]             ` <46FA285B.5060709-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-09-26 17:31               ` Christoph Lameter
     [not found]                 ` <Pine.LNX.4.64.0709261030270.15435-RYO/mD75kfhx2SFC9UQUAuF7EQX82lMiAL8bYrjMMd8@public.gmane.org>
2007-09-27  8:25                   ` Pavel Emelyanov
2007-10-01 11:55       ` Balbir Singh
     [not found]         ` <4700E03B.6000102-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 12:13           ` Pavel Emelyanov
     [not found]             ` <4700E477.2060607-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 12:32               ` Balbir Singh
     [not found]                 ` <4700E8F2.7000206-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 12:57                   ` Pavel Emelyanov
     [not found]                     ` <4700EEAE.1090208-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 13:03                       ` Balbir Singh
2007-09-25 14:19   ` [PATCH 2/5] Generic notifiers for SLUB events Pavel Emelyanov
     [not found]     ` <46F918D9.3020406-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 13:05       ` Balbir Singh
     [not found]         ` <4700F083.1070706-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 13:07           ` Pavel Emelyanov
     [not found]             ` <4700F120.2070302-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 20:39               ` Christoph Lameter
2007-09-25 14:22   ` [PATCH 3/5] Switch caches notification dynamically Pavel Emelyanov
     [not found]     ` <46F919BB.2000701-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-09-25 21:48       ` Christoph Lameter
     [not found]         ` <Pine.LNX.4.64.0709251447560.5072-RYO/mD75kfhx2SFC9UQUAuF7EQX82lMiAL8bYrjMMd8@public.gmane.org>
2007-09-26  9:39           ` Pavel Emelyanov
     [not found]             ` <46FA28C9.9060101-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-09-26 17:30               ` Christoph Lameter
     [not found]                 ` <Pine.LNX.4.64.0709261029300.15435-RYO/mD75kfhx2SFC9UQUAuF7EQX82lMiAL8bYrjMMd8@public.gmane.org>
2007-09-27  8:24                   ` Pavel Emelyanov
2007-10-01 13:15       ` Balbir Singh
     [not found]         ` <4700F2E8.5050904-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 13:19           ` Pavel Emelyanov
2007-10-01 13:21           ` Pavel Emelyanov
     [not found]             ` <4700F476.4070806-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 13:38               ` Balbir Singh
     [not found]                 ` <4700F85B.5090902-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 13:45                   ` Pavel Emelyanov
     [not found]                     ` <4700FA0A.5040707-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 14:14                       ` Balbir Singh
     [not found]                         ` <470100D0.7030700-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 15:45                           ` Pavel Emelyanov
2007-10-01 20:39                           ` Christoph Lameter
2007-09-25 14:24   ` [PATCH 4/5] Setup the control group Pavel Emelyanov
     [not found]     ` <46F91A1E.2060303-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 13:48       ` Balbir Singh
     [not found]         ` <4700FAAC.2050004-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 13:51           ` Pavel Emelyanov
     [not found]             ` <4700FB72.5070409-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 14:16               ` Balbir Singh
     [not found]                 ` <4701013A.3010307-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 14:17                   ` Pavel Emelyanov
     [not found]                     ` <47010169.5040102-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 14:21                       ` Balbir Singh
     [not found]                         ` <47010276.8060802-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 14:27                           ` Pavel Emelyanov
2007-10-01 15:50                           ` [Devel] " Paul Menage
     [not found]                             ` <6599ad830710010850q660d042av9fa5a461d3c3e445-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2007-10-01 15:53                               ` Balbir Singh
     [not found]                                 ` <47011812.2010406-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 16:04                                   ` Paul Menage
2007-10-01 15:52                   ` Paul Menage
2007-09-25 14:26   ` [PATCH 5/5] Account for the slub objects Pavel Emelyanov
     [not found]     ` <46F91A8A.9000001-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 14:07       ` Balbir Singh
     [not found]         ` <4700FF1F.7060604-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 14:10           ` Pavel Emelyanov
     [not found]             ` <4700FFC3.7050005-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-01 20:41               ` Christoph Lameter
     [not found]                 ` <Pine.LNX.4.64.0710011341000.19779-RYO/mD75kfhx2SFC9UQUAuF7EQX82lMiAL8bYrjMMd8@public.gmane.org>
2007-10-02 12:44                   ` Pavel Emelyanov
     [not found]                     ` <47023D18.3090304-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-02 18:04                       ` Christoph Lameter
     [not found]                         ` <Pine.LNX.4.64.0710021104000.30559-RYO/mD75kfhx2SFC9UQUAuF7EQX82lMiAL8bYrjMMd8@public.gmane.org>
2007-10-03  7:29                           ` Pavel Emelyanov
2007-10-01 14:12   ` [PATCH 0/5] Kernel memory accounting container (v5) Balbir Singh
     [not found]     ` <4701006B.6050809-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2007-10-01 15:43       ` Pavel Emelyanov
2007-10-01 16:32   ` [Devel] " Paul Menage
     [not found]     ` <6599ad830710010932t150aba2eid18864a90f169c64-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2007-10-02 12:51       ` Pavel Emelyanov
     [not found]         ` <47023EBE.7000708-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2007-10-05  7:11           ` Paul Menage
     [not found]             ` <6599ad830710050011x68a80013w3b60d663e2c087a-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2007-10-05 13:17               ` Pavel Emelyanov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=46F91841.9070708@openvz.org \
    --to=xemul-gefaqzzx7r8dnm+yrofe0a@public.gmane.org \
    --cc=balbir-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org \
    --cc=clameter-sJ/iWh9BUns@public.gmane.org \
    --cc=containers-qjLDD68F18O7TbgM5vRIOg@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.