From: Mel Gorman <mgorman-l3A5Bk7waGM@public.gmane.org>
To: Glauber Costa <glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
Cc: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Andrew Morton
<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
Greg Thelen <gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org,
Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>,
Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
Dave Chinner <dchinner-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH v4 05/31] dcache: remove dentries from LRU before putting on dispose list
Date: Tue, 30 Apr 2013 15:14:56 +0100 [thread overview]
Message-ID: <20130430141456.GE6415@suse.de> (raw)
In-Reply-To: <1367018367-11278-6-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
On Sat, Apr 27, 2013 at 03:19:01AM +0400, Glauber Costa wrote:
> From: Dave Chinner <dchinner-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>
> One of the big problems with modifying the way the dcache shrinker
> and LRU implementation works is that the LRU is abused in several
> ways. One of these is shrink_dentry_list().
>
> Basically, we can move a dentry off the LRU onto a different list
> without doing any accounting changes, and then use dentry_lru_prune()
> to remove it from what-ever list it is now on to do the LRU
> accounting at that point.
>
> This makes it -really hard- to change the LRU implementation. The
> use of the per-sb LRU lock serialises movement of the dentries
> between the different lists and the removal of them, and this is the
> only reason that it works. If we want to break up the dentry LRU
> lock and lists into, say, per-node lists, we remove the only
> serialisation that allows this lru list/dispose list abuse to work.
>
> To make this work effectively, the dispose list has to be isolated
> from the LRU list - dentries have to be removed from the LRU
> *before* being placed on the dispose list. This means that the LRU
> accounting and isolation is completed before disposal is started,
> and that means we can change the LRU implementation freely in
> future.
>
> This means that dentries *must* be marked with DCACHE_SHRINK_LIST
> when they are placed on the dispose list so that we don't think that
> parent dentries found in try_prune_one_dentry() are on the LRU when
> the are actually on the dispose list. This would result in
> accounting the dentry to the LRU a second time. Hence
> dentry_lru_prune() has to handle the DCACHE_SHRINK_LIST case
> differently because the dentry isn't on the LRU list.
>
> [ v2: don't decrement nr unused twice, spotted by Sha Zhengju ]
> Signed-off-by: Dave Chinner <dchinner-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Glauber Costa <glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
Acked-by: Mel Gorman <mgorman-l3A5Bk7waGM@public.gmane.org>
--
Mel Gorman
SUSE Labs
WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de>
To: Glauber Costa <glommer@openvz.org>
Cc: linux-mm@kvack.org, cgroups@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
Greg Thelen <gthelen@google.com>,
kamezawa.hiroyu@jp.fujitsu.com, Michal Hocko <mhocko@suse.cz>,
Johannes Weiner <hannes@cmpxchg.org>,
Dave Chinner <dchinner@redhat.com>
Subject: Re: [PATCH v4 05/31] dcache: remove dentries from LRU before putting on dispose list
Date: Tue, 30 Apr 2013 15:14:56 +0100 [thread overview]
Message-ID: <20130430141456.GE6415@suse.de> (raw)
In-Reply-To: <1367018367-11278-6-git-send-email-glommer@openvz.org>
On Sat, Apr 27, 2013 at 03:19:01AM +0400, Glauber Costa wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> One of the big problems with modifying the way the dcache shrinker
> and LRU implementation works is that the LRU is abused in several
> ways. One of these is shrink_dentry_list().
>
> Basically, we can move a dentry off the LRU onto a different list
> without doing any accounting changes, and then use dentry_lru_prune()
> to remove it from what-ever list it is now on to do the LRU
> accounting at that point.
>
> This makes it -really hard- to change the LRU implementation. The
> use of the per-sb LRU lock serialises movement of the dentries
> between the different lists and the removal of them, and this is the
> only reason that it works. If we want to break up the dentry LRU
> lock and lists into, say, per-node lists, we remove the only
> serialisation that allows this lru list/dispose list abuse to work.
>
> To make this work effectively, the dispose list has to be isolated
> from the LRU list - dentries have to be removed from the LRU
> *before* being placed on the dispose list. This means that the LRU
> accounting and isolation is completed before disposal is started,
> and that means we can change the LRU implementation freely in
> future.
>
> This means that dentries *must* be marked with DCACHE_SHRINK_LIST
> when they are placed on the dispose list so that we don't think that
> parent dentries found in try_prune_one_dentry() are on the LRU when
> the are actually on the dispose list. This would result in
> accounting the dentry to the LRU a second time. Hence
> dentry_lru_prune() has to handle the DCACHE_SHRINK_LIST case
> differently because the dentry isn't on the LRU list.
>
> [ v2: don't decrement nr unused twice, spotted by Sha Zhengju ]
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> Signed-off-by: Glauber Costa <glommer@openvz.org>
Acked-by: Mel Gorman <mgorman@suse.de>
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2013-04-30 14:14 UTC|newest]
Thread overview: 105+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-04-26 23:18 [PATCH v4 00/31] kmemcg shrinkers Glauber Costa
2013-04-26 23:18 ` [PATCH v4 01/31] super: fix calculation of shrinkable objects for small numbers Glauber Costa
2013-04-30 13:03 ` Mel Gorman
2013-04-26 23:18 ` [PATCH v4 02/31] vmscan: take at least one pass with shrinkers Glauber Costa
[not found] ` <1367018367-11278-3-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 13:22 ` Mel Gorman
2013-04-30 13:22 ` Mel Gorman
2013-04-30 13:31 ` Glauber Costa
[not found] ` <517FC7B4.5030101-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org>
2013-04-30 15:37 ` Mel Gorman
2013-04-30 15:37 ` Mel Gorman
2013-05-07 13:35 ` Glauber Costa
[not found] ` <1367018367-11278-1-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-26 23:18 ` [PATCH v4 03/31] dcache: convert dentry_stat.nr_unused to per-cpu counters Glauber Costa
2013-04-26 23:18 ` Glauber Costa
2013-04-30 13:37 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 04/31] dentry: move to per-sb LRU locks Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-5-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 14:01 ` Mel Gorman
2013-04-30 14:01 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 05/31] dcache: remove dentries from LRU before putting on dispose list Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-6-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 14:14 ` Mel Gorman [this message]
2013-04-30 14:14 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 06/31] mm: new shrinker API Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-7-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 14:40 ` Mel Gorman
2013-04-30 14:40 ` Mel Gorman
2013-04-30 15:03 ` Glauber Costa
2013-04-30 15:32 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 07/31] shrinker: convert superblock shrinkers to new API Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-30 14:49 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 08/31] list: add a new LRU list type Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-30 15:18 ` Mel Gorman
2013-04-30 16:01 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 09/31] inode: convert inode lru list to generic lru list code Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-10-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 15:46 ` Mel Gorman
2013-04-30 15:46 ` Mel Gorman
2013-05-07 13:47 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 10/31] dcache: convert to use new lru list infrastructure Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-11-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 16:04 ` Mel Gorman
2013-04-30 16:04 ` Mel Gorman
2013-04-30 16:13 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 11/31] list_lru: per-node " Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-12-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 16:33 ` Mel Gorman
2013-04-30 16:33 ` Mel Gorman
2013-04-30 21:44 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 12/31] shrinker: add node awareness Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-13-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 16:35 ` Mel Gorman
2013-04-30 16:35 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 13/31] fs: convert inode and dentry shrinking to be node aware Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-14-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 17:39 ` Mel Gorman
2013-04-30 17:39 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 14/31] xfs: convert buftarg LRU to generic code Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 15/31] xfs: convert dquot cache lru to list_lru Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 16/31] fs: convert fs shrinkers to new scan/count API Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 17/31] drivers: convert shrinkers to new count/scan API Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-30 21:53 ` Mel Gorman
[not found] ` <20130430215355.GN6415-l3A5Bk7waGM@public.gmane.org>
2013-04-30 22:00 ` Kent Overstreet
2013-04-30 22:00 ` Kent Overstreet
[not found] ` <20130430220050.GK9931-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2013-05-02 9:37 ` Mel Gorman
2013-05-02 9:37 ` Mel Gorman
2013-05-02 13:37 ` Glauber Costa
2013-05-01 15:26 ` Daniel Vetter
2013-05-02 9:31 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 18/31] shrinker: convert remaining shrinkers to " Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 19/31] hugepage: convert huge zero page shrinker to new shrinker API Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 20/31] shrinker: Kill old ->shrink API Glauber Costa
2013-04-26 23:19 ` Glauber Costa
[not found] ` <1367018367-11278-21-git-send-email-glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
2013-04-30 21:57 ` Mel Gorman
2013-04-30 21:57 ` Mel Gorman
2013-04-26 23:19 ` [PATCH v4 21/31] vmscan: also shrink slab in memcg pressure Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 22/31] memcg,list_lru: duplicate LRUs upon kmemcg creation Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 23/31] lru: add an element to a memcg list Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 24/31] list_lru: per-memcg walks Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 25/31] memcg: per-memcg kmem shrinking Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 26/31] memcg: scan cache objects hierarchically Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 27/31] super: targeted memcg reclaim Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 28/31] memcg: move initialization to memcg creation Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 29/31] vmpressure: in-kernel notifications Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 30/31] memcg: reap dead memcgs upon global memory pressure Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-26 23:19 ` [PATCH v4 31/31] memcg: debugging facility to access dangling memcgs Glauber Costa
2013-04-26 23:19 ` Glauber Costa
2013-04-30 22:47 ` [PATCH v4 00/31] kmemcg shrinkers Mel Gorman
2013-05-01 9:05 ` Mel Gorman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130430141456.GE6415@suse.de \
--to=mgorman-l3a5bk7wagm@public.gmane.org \
--cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
--cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=dchinner-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
--cc=glommer-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org \
--cc=gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
--cc=hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org \
--cc=kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org \
--cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
--cc=mhocko-AlSwsSmVLrQ@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.