linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Glauber Costa <glommer@parallels.com>
To: <linux-mm@kvack.org>
Cc: <cgroups@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Michal Hocko <mhocko@suse.cz>,
	Johannes Weiner <hannes@cmpxchg.org>,
	<kamezawa.hiroyu@jp.fujitsu.com>,
	Dave Shrinnker <david@fromorbit.com>,
	<linux-fsdevel@vger.kernel.org>
Subject: [PATCH 0/7] memcg targeted shrinking
Date: Fri,  8 Feb 2013 17:07:30 +0400	[thread overview]
Message-ID: <1360328857-28070-1-git-send-email-glommer@parallels.com> (raw)

This patchset implements targeted shrinking for memcg when kmem limits are
present. So far, we've been accounting kernel objects but failing allocations
when short of memory. This is because our only option would be to call the
global shrinker, depleting objects from all caches and breaking isolation.

This patchset builds upon the recent work from David Chinner
(http://oss.sgi.com/archives/xfs/2012-11/msg00643.html) to implement NUMA
aware per-node LRUs. I build heavily on its API, and its presence is implied.

The main idea is to associate per-memcg lists with each of the LRUs. The main
LRU still provides a single entry point and when adding or removing an element
from the LRU, we use the page information to figure out which memcg it belongs
to and relay it to the right list.

This patchset is still not perfect, and some uses cases still need to be
dealt with. But I wanted to get this out in the open sooner rather than
later. In particular, I have the following (noncomprehensive) todo list:

TODO:
* shrink dead memcgs when global pressure kicks in.
* balance global reclaim among memcgs.
* improve testing and reliability (I am still seeing some stalls in some cases)

Glauber Costa (7):
  vmscan: also shrink slab in memcg pressure
  memcg,list_lru: duplicate LRUs upon kmemcg creation
  lru: add an element to a memcg list
  list_lru: also include memcg lists in counts and scans
  list_lru: per-memcg walks
  super: targeted memcg reclaim
  memcg: per-memcg kmem shrinking

 fs/dcache.c                |   7 +-
 fs/inode.c                 |   6 +-
 fs/internal.h              |   5 +-
 fs/super.c                 |  37 ++++--
 include/linux/list_lru.h   |  81 +++++++++++-
 include/linux/memcontrol.h |  34 +++++
 include/linux/shrinker.h   |   4 +
 include/linux/swap.h       |   2 +
 lib/list_lru.c             | 301 ++++++++++++++++++++++++++++++++++++++-------
 mm/memcontrol.c            | 271 ++++++++++++++++++++++++++++++++++++++--
 mm/slab_common.c           |   1 -
 mm/vmscan.c                |  78 +++++++++++-
 12 files changed, 747 insertions(+), 80 deletions(-)

-- 
1.8.1


             reply	other threads:[~2013-02-08 13:07 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-02-08 13:07 Glauber Costa [this message]
2013-02-08 13:07 ` [PATCH 1/7] vmscan: also shrink slab in memcg pressure Glauber Costa
2013-02-15  1:27   ` Greg Thelen
2013-02-15 10:46     ` Glauber Costa
2013-02-15  8:37   ` Kamezawa Hiroyuki
     [not found]     ` <511DF3CB.7020206-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2013-02-15 10:30       ` Glauber Costa
2013-02-08 13:07 ` [PATCH 2/7] memcg,list_lru: duplicate LRUs upon kmemcg creation Glauber Costa
2013-02-15  1:31   ` Greg Thelen
     [not found]     ` <xr934nhenz18.fsf-aSPv4SP+Du0KgorLzL7FmE7CuiCeIGUxQQ4Iyu8u01E@public.gmane.org>
2013-02-15 10:54       ` Glauber Costa
2013-02-20  7:46         ` Greg Thelen
     [not found]   ` <1360328857-28070-3-git-send-email-glommer-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org>
2013-02-15  9:21     ` Kamezawa Hiroyuki
2013-02-15 10:36       ` Glauber Costa
2013-02-08 13:07 ` [PATCH 3/7] lru: add an element to a memcg list Glauber Costa
2013-02-15  1:32   ` Greg Thelen
     [not found]     ` <xr93txpemkeo.fsf-aSPv4SP+Du0KgorLzL7FmE7CuiCeIGUxQQ4Iyu8u01E@public.gmane.org>
2013-02-15 10:57       ` Glauber Costa
2013-02-08 13:07 ` [PATCH 4/7] list_lru: also include memcg lists in counts and scans Glauber Costa
2013-02-08 13:07 ` [PATCH 5/7] list_lru: per-memcg walks Glauber Costa
2013-02-08 13:07 ` [PATCH 6/7] super: targeted memcg reclaim Glauber Costa
2013-02-08 13:07 ` [PATCH 7/7] memcg: per-memcg kmem shrinking Glauber Costa
     [not found] ` <1360328857-28070-1-git-send-email-glommer-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org>
2013-02-15  1:28   ` [PATCH 0/7] memcg targeted shrinking Greg Thelen
     [not found]     ` <xr93ip5unz52.fsf-aSPv4SP+Du0KgorLzL7FmE7CuiCeIGUxQQ4Iyu8u01E@public.gmane.org>
2013-02-15 10:42       ` Glauber Costa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1360328857-28070-1-git-send-email-glommer@parallels.com \
    --to=glommer@parallels.com \
    --cc=akpm@linux-foundation.org \
    --cc=cgroups@vger.kernel.org \
    --cc=david@fromorbit.com \
    --cc=hannes@cmpxchg.org \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.cz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).