From: Johannes Weiner <hannes@cmpxchg.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: linux-mm@kvack.org, Andi Kleen <andi@firstfloor.org>,
Andrea Arcangeli <aarcange@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Greg Thelen <gthelen@google.com>,
Christoph Hellwig <hch@infradead.org>,
Hugh Dickins <hughd@google.com>, Jan Kara <jack@suse.cz>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Mel Gorman <mgorman@suse.de>, Minchan Kim <minchan.kim@gmail.com>,
Rik van Riel <riel@redhat.com>,
Michel Lespinasse <walken@google.com>,
Seth Jennings <sjenning@linux.vnet.ibm.com>,
Roman Gushchin <klamm@yandex-team.ru>, metin d <metdos@yahoo.com>,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org
Subject: Re: [patch 10/10] mm: workingset: keep shadow entries in check
Date: Mon, 3 Jun 2013 11:20:32 -0400 [thread overview]
Message-ID: <20130603152032.GF15576@cmpxchg.org> (raw)
In-Reply-To: <20130603082533.GH5910@twins.programming.kicks-ass.net>
On Mon, Jun 03, 2013 at 10:25:33AM +0200, Peter Zijlstra wrote:
> On Thu, May 30, 2013 at 02:04:06PM -0400, Johannes Weiner wrote:
> > Previously, page cache radix tree nodes were freed after reclaim
> > emptied out their page pointers. But now reclaim stores shadow
> > entries in their place, which are only reclaimed when the inodes
> > themselves are reclaimed. This is problematic for bigger files that
> > are still in use after they have a significant amount of their cache
> > reclaimed, without any of those pages actually refaulting. The shadow
> > entries will just sit there and waste memory. In the worst case, the
> > shadow entries will accumulate until the machine runs out of memory.
> >
>
> Can't we simply prune all refault entries that have a distance larger
> than the memory size? Then we must assume that no refault entry means
> its too old, which I think is a fair assumption.
Two workloads bound to two nodes might not push pages through the LRUs
at the same pace, so a distance might be bigger than memory due to the
faster moving node, yet still be a hit in the slower moving one. We
can't really know until we evaluate it on a per-zone basis.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Johannes Weiner <hannes@cmpxchg.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: linux-mm@kvack.org, Andi Kleen <andi@firstfloor.org>,
Andrea Arcangeli <aarcange@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Greg Thelen <gthelen@google.com>,
Christoph Hellwig <hch@infradead.org>,
Hugh Dickins <hughd@google.com>, Jan Kara <jack@suse.cz>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Mel Gorman <mgorman@suse.de>, Minchan Kim <minchan.kim@gmail.com>,
Rik van Riel <riel@redhat.com>,
Michel Lespinasse <walken@google.com>,
Seth Jennings <sjenning@linux.vnet.ibm.com>,
Roman Gushchin <klamm@yandex-team.ru>, metin d <metdos@yahoo.com>,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org
Subject: Re: [patch 10/10] mm: workingset: keep shadow entries in check
Date: Mon, 3 Jun 2013 11:20:32 -0400 [thread overview]
Message-ID: <20130603152032.GF15576@cmpxchg.org> (raw)
In-Reply-To: <20130603082533.GH5910@twins.programming.kicks-ass.net>
On Mon, Jun 03, 2013 at 10:25:33AM +0200, Peter Zijlstra wrote:
> On Thu, May 30, 2013 at 02:04:06PM -0400, Johannes Weiner wrote:
> > Previously, page cache radix tree nodes were freed after reclaim
> > emptied out their page pointers. But now reclaim stores shadow
> > entries in their place, which are only reclaimed when the inodes
> > themselves are reclaimed. This is problematic for bigger files that
> > are still in use after they have a significant amount of their cache
> > reclaimed, without any of those pages actually refaulting. The shadow
> > entries will just sit there and waste memory. In the worst case, the
> > shadow entries will accumulate until the machine runs out of memory.
> >
>
> Can't we simply prune all refault entries that have a distance larger
> than the memory size? Then we must assume that no refault entry means
> its too old, which I think is a fair assumption.
Two workloads bound to two nodes might not push pages through the LRUs
at the same pace, so a distance might be bigger than memory due to the
faster moving node, yet still be a hit in the slower moving one. We
can't really know until we evaluate it on a per-zone basis.
next prev parent reply other threads:[~2013-06-03 15:20 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-30 18:03 [patch 00/10] mm: thrash detection-based file cache sizing Johannes Weiner
2013-05-30 18:03 ` Johannes Weiner
2013-05-30 18:03 ` [patch 01/10] mm: page_alloc: zone round-robin allocator Johannes Weiner
2013-05-30 18:03 ` Johannes Weiner
2013-05-30 18:03 ` [patch 02/10] lib: radix-tree: radix_tree_delete_item() Johannes Weiner
2013-05-30 18:03 ` Johannes Weiner
2013-05-30 18:03 ` [patch 03/10] mm: shmem: save one radix tree lookup when truncating swapped pages Johannes Weiner
2013-05-30 18:03 ` Johannes Weiner
2013-05-30 18:04 ` [patch 04/10] mm: filemap: move radix tree hole searching here Johannes Weiner
2013-05-30 18:04 ` Johannes Weiner
2013-05-30 18:04 ` [patch 05/10] mm + fs: prepare for non-page entries in page cache radix trees Johannes Weiner
2013-05-30 18:04 ` Johannes Weiner
2013-05-30 18:04 ` [patch 06/10] mm + fs: store shadow entries in page cache Johannes Weiner
2013-05-30 18:04 ` Johannes Weiner
2013-05-30 18:04 ` [patch 07/10] mm + fs: provide refault distance to page cache allocations Johannes Weiner
2013-05-30 18:04 ` Johannes Weiner
2013-05-30 18:04 ` [patch 08/10] mm: make global_dirtyable_memory() available to other mm code Johannes Weiner
2013-05-30 18:04 ` Johannes Weiner
2013-05-30 18:04 ` [patch 09/10] mm: thrash detection-based file cache sizing Johannes Weiner
2013-05-30 18:04 ` Johannes Weiner
2013-06-07 14:16 ` Roman Gushchin
2013-06-07 14:16 ` Roman Gushchin
2013-06-07 17:36 ` Johannes Weiner
2013-06-07 17:36 ` Johannes Weiner
2013-05-30 18:04 ` [patch 10/10] mm: workingset: keep shadow entries in check Johannes Weiner
2013-05-30 18:04 ` Johannes Weiner
2013-06-03 8:22 ` Peter Zijlstra
2013-06-03 8:22 ` Peter Zijlstra
2013-06-03 15:01 ` Johannes Weiner
2013-06-03 15:01 ` Johannes Weiner
2013-06-03 17:10 ` Peter Zijlstra
2013-06-03 17:10 ` Peter Zijlstra
2013-06-06 18:31 ` Johannes Weiner
2013-06-06 18:31 ` Johannes Weiner
2013-06-03 8:25 ` Peter Zijlstra
2013-06-03 8:25 ` Peter Zijlstra
2013-06-03 15:20 ` Johannes Weiner [this message]
2013-06-03 15:20 ` Johannes Weiner
2013-06-03 17:15 ` Peter Zijlstra
2013-06-03 17:15 ` Peter Zijlstra
2013-06-03 18:12 ` Johannes Weiner
2013-06-03 18:12 ` Johannes Weiner
2013-06-03 18:52 ` Peter Zijlstra
2013-06-03 18:52 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130603152032.GF15576@cmpxchg.org \
--to=hannes@cmpxchg.org \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=andi@firstfloor.org \
--cc=gthelen@google.com \
--cc=hch@infradead.org \
--cc=hughd@google.com \
--cc=jack@suse.cz \
--cc=klamm@yandex-team.ru \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=metdos@yahoo.com \
--cc=mgorman@suse.de \
--cc=minchan.kim@gmail.com \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=sjenning@linux.vnet.ibm.com \
--cc=walken@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.