All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bob Liu <bob.liu@oracle.com>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Andi Kleen <andi@firstfloor.org>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Christoph Hellwig <hch@infradead.org>,
	Dave Chinner <david@fromorbit.com>,
	Greg Thelen <gthelen@google.com>, Hugh Dickins <hughd@google.com>,
	Jan Kara <jack@suse.cz>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Luigi Semenzato <semenzato@google.com>,
	Mel Gorman <mgorman@suse.de>, Metin Doslu <metin@citusdata.com>,
	Michel Lespinasse <walken@google.com>,
	Minchan Kim <minchan.kim@gmail.com>,
	Ozgun Erdogan <ozgun@citusdata.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Rik van Riel <riel@redhat.com>,
	Roman Gushchin <klamm@yandex-team.ru>,
	Ryan Mallon <rmallon@gmail.com>, Tejun Heo <tj@kernel.org>,
	Vlastimil Babka <vbabka@suse.cz>, linux-mm@kvack.org,
Subject: Re: [patch 9/9] mm: keep page cache radix tree nodes in check
Date: Wed, 15 Jan 2014 13:55:01 +0800	[thread overview]
Message-ID: <52D622B5.6070203@oracle.com> (raw)
In-Reply-To: <1389377443-11755-10-git-send-email-hannes@cmpxchg.org>

Hi Johannes,

On 01/11/2014 02:10 AM, Johannes Weiner wrote:
> Previously, page cache radix tree nodes were freed after reclaim
> emptied out their page pointers.  But now reclaim stores shadow
> entries in their place, which are only reclaimed when the inodes
> themselves are reclaimed.  This is problematic for bigger files that
> are still in use after they have a significant amount of their cache
> reclaimed, without any of those pages actually refaulting.  The shadow
> entries will just sit there and waste memory.  In the worst case, the
> shadow entries will accumulate until the machine runs out of memory.
> 

I have one more question. It seems that other algorithm only remember
history information of a limit number of evicted pages where the number
is usually the same as the total cache or memory size.
But in your patch, I didn't see a preferred value that how many evicted
pages' history information should be recorded. It all depends on the
workingset_shadow_shrinker?

Thanks,
-Bob

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Bob Liu <bob.liu@oracle.com>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Andi Kleen <andi@firstfloor.org>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Christoph Hellwig <hch@infradead.org>,
	Dave Chinner <david@fromorbit.com>,
	Greg Thelen <gthelen@google.com>, Hugh Dickins <hughd@google.com>,
	Jan Kara <jack@suse.cz>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Luigi Semenzato <semenzato@google.com>,
	Mel Gorman <mgorman@suse.de>, Metin Doslu <metin@citusdata.com>,
	Michel Lespinasse <walken@google.com>,
	Minchan Kim <minchan.kim@gmail.com>,
	Ozgun Erdogan <ozgun@citusdata.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Rik van Riel <riel@redhat.com>,
	Roman Gushchin <klamm@yandex-team.ru>,
	Ryan Mallon <rmallon@gmail.com>, Tejun Heo <tj@kernel.org>,
	Vlastimil Babka <vbabka@suse.cz>,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [patch 9/9] mm: keep page cache radix tree nodes in check
Date: Wed, 15 Jan 2014 13:55:01 +0800	[thread overview]
Message-ID: <52D622B5.6070203@oracle.com> (raw)
In-Reply-To: <1389377443-11755-10-git-send-email-hannes@cmpxchg.org>

Hi Johannes,

On 01/11/2014 02:10 AM, Johannes Weiner wrote:
> Previously, page cache radix tree nodes were freed after reclaim
> emptied out their page pointers.  But now reclaim stores shadow
> entries in their place, which are only reclaimed when the inodes
> themselves are reclaimed.  This is problematic for bigger files that
> are still in use after they have a significant amount of their cache
> reclaimed, without any of those pages actually refaulting.  The shadow
> entries will just sit there and waste memory.  In the worst case, the
> shadow entries will accumulate until the machine runs out of memory.
> 

I have one more question. It seems that other algorithm only remember
history information of a limit number of evicted pages where the number
is usually the same as the total cache or memory size.
But in your patch, I didn't see a preferred value that how many evicted
pages' history information should be recorded. It all depends on the
workingset_shadow_shrinker?

Thanks,
-Bob

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Bob Liu <bob.liu@oracle.com>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Andi Kleen <andi@firstfloor.org>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Christoph Hellwig <hch@infradead.org>,
	Dave Chinner <david@fromorbit.com>,
	Greg Thelen <gthelen@google.com>, Hugh Dickins <hughd@google.com>,
	Jan Kara <jack@suse.cz>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Luigi Semenzato <semenzato@google.com>,
	Mel Gorman <mgorman@suse.de>, Metin Doslu <metin@citusdata.com>,
	Michel Lespinasse <walken@google.com>,
	Minchan Kim <minchan.kim@gmail.com>,
	Ozgun Erdogan <ozgun@citusdata.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Rik van Riel <riel@redhat.com>,
	Roman Gushchin <klamm@yandex-team.ru>,
	Ryan Mallon <rmallon@gmail.com>, Tejun Heo <tj@kernel.org>,
	Vlastimil Babka <vbabka@suse.cz>,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [patch 9/9] mm: keep page cache radix tree nodes in check
Date: Wed, 15 Jan 2014 13:55:01 +0800	[thread overview]
Message-ID: <52D622B5.6070203@oracle.com> (raw)
In-Reply-To: <1389377443-11755-10-git-send-email-hannes@cmpxchg.org>

Hi Johannes,

On 01/11/2014 02:10 AM, Johannes Weiner wrote:
> Previously, page cache radix tree nodes were freed after reclaim
> emptied out their page pointers.  But now reclaim stores shadow
> entries in their place, which are only reclaimed when the inodes
> themselves are reclaimed.  This is problematic for bigger files that
> are still in use after they have a significant amount of their cache
> reclaimed, without any of those pages actually refaulting.  The shadow
> entries will just sit there and waste memory.  In the worst case, the
> shadow entries will accumulate until the machine runs out of memory.
> 

I have one more question. It seems that other algorithm only remember
history information of a limit number of evicted pages where the number
is usually the same as the total cache or memory size.
But in your patch, I didn't see a preferred value that how many evicted
pages' history information should be recorded. It all depends on the
workingset_shadow_shrinker?

Thanks,
-Bob

  parent reply	other threads:[~2014-01-15  5:55 UTC|newest]

Thread overview: 126+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-01-10 18:10 [patch 0/9] mm: thrash detection-based file cache sizing v8 Johannes Weiner
2014-01-10 18:10 ` Johannes Weiner
2014-01-10 18:10 ` [patch 1/9] fs: cachefiles: use add_to_page_cache_lru() Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-13  1:17   ` Minchan Kim
2014-01-13  1:17     ` Minchan Kim
2014-01-10 18:10 ` [patch 2/9] lib: radix-tree: radix_tree_delete_item() Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-10 18:10 ` [patch 3/9] mm: shmem: save one radix tree lookup when truncating swapped pages Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-10 18:25   ` Rik van Riel
2014-01-10 18:25     ` Rik van Riel
2014-01-10 18:25     ` Rik van Riel
2014-01-10 18:10 ` [patch 4/9] mm: filemap: move radix tree hole searching here Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-10 19:22   ` Rik van Riel
2014-01-10 19:22     ` Rik van Riel
2014-01-10 19:22     ` Rik van Riel
2014-01-13  1:25   ` Minchan Kim
2014-01-13  1:25     ` Minchan Kim
2014-01-10 18:10 ` [patch 5/9] mm + fs: prepare for non-page entries in page cache radix trees Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-10 19:39   ` Rik van Riel
2014-01-10 19:39     ` Rik van Riel
2014-01-10 19:39     ` Rik van Riel
2014-01-13  2:01   ` Minchan Kim
2014-01-13  2:01     ` Minchan Kim
2014-01-22 17:47     ` Johannes Weiner
2014-01-22 17:47       ` Johannes Weiner
2014-01-23  5:07       ` Minchan Kim
2014-01-23  5:07         ` Minchan Kim
2014-02-12 14:00   ` Mel Gorman
2014-02-12 14:00     ` Mel Gorman
2014-03-12  1:15     ` Johannes Weiner
2014-03-12  1:15       ` Johannes Weiner
2014-01-10 18:10 ` [patch 6/9] mm + fs: store shadow entries in page cache Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-10 22:30   ` Rik van Riel
2014-01-10 22:30     ` Rik van Riel
2014-01-10 22:30     ` Rik van Riel
2014-01-13  2:18   ` Minchan Kim
2014-01-13  2:18     ` Minchan Kim
2014-01-10 18:10 ` [patch 7/9] mm: thrash detection-based file cache sizing Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-10 22:51   ` Rik van Riel
2014-01-10 22:51     ` Rik van Riel
2014-01-10 22:51     ` Rik van Riel
2014-01-13  2:42   ` Minchan Kim
2014-01-13  2:42     ` Minchan Kim
2014-01-14  1:01   ` Bob Liu
2014-01-14  1:01     ` Bob Liu
2014-01-14  1:01     ` Bob Liu
2014-01-14 19:16     ` Johannes Weiner
2014-01-14 19:16       ` Johannes Weiner
2014-01-15  2:57       ` Bob Liu
2014-01-15  2:57         ` Bob Liu
2014-01-15  2:57         ` Bob Liu
2014-01-15  3:52         ` Zhang Yanfei
2014-01-15  3:52           ` Zhang Yanfei
2014-01-16 21:17         ` Johannes Weiner
2014-01-16 21:17           ` Johannes Weiner
2014-01-10 18:10 ` [patch 8/9] lib: radix_tree: tree node interface Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-10 22:57   ` Rik van Riel
2014-01-10 22:57     ` Rik van Riel
2014-01-10 22:57     ` Rik van Riel
2014-01-10 18:10 ` [patch 9/9] mm: keep page cache radix tree nodes in check Johannes Weiner
2014-01-10 18:10   ` Johannes Weiner
2014-01-10 23:09   ` Rik van Riel
2014-01-10 23:09     ` Rik van Riel
2014-01-10 23:09     ` Rik van Riel
2014-01-13  7:39   ` Minchan Kim
2014-01-13  7:39     ` Minchan Kim
2014-01-14  5:40     ` Minchan Kim
2014-01-14  5:40       ` Minchan Kim
2014-01-22 18:42     ` Johannes Weiner
2014-01-22 18:42       ` Johannes Weiner
2014-01-23  5:20       ` Minchan Kim
2014-01-23  5:20         ` Minchan Kim
2014-01-23 19:22         ` Johannes Weiner
2014-01-23 19:22           ` Johannes Weiner
2014-01-27  2:31           ` Minchan Kim
2014-01-27  2:31             ` Minchan Kim
2014-01-15  5:55   ` Bob Liu [this message]
2014-01-15  5:55     ` Bob Liu
2014-01-15  5:55     ` Bob Liu
2014-01-16 22:09     ` Johannes Weiner
2014-01-16 22:09       ` Johannes Weiner
2014-01-17  0:05   ` Dave Chinner
2014-01-17  0:05     ` Dave Chinner
2014-01-20 23:17     ` Johannes Weiner
2014-01-20 23:17       ` Johannes Weiner
2014-01-21  3:03       ` Dave Chinner
2014-01-21  3:03         ` Dave Chinner
2014-01-21  5:50         ` Johannes Weiner
2014-01-21  5:50           ` Johannes Weiner
2014-01-22  3:06           ` Dave Chinner
2014-01-22  3:06             ` Dave Chinner
2014-01-22  6:57             ` Johannes Weiner
2014-01-22  6:57               ` Johannes Weiner
2014-01-22 18:48               ` Johannes Weiner
2014-01-22 18:48                 ` Johannes Weiner
2014-01-23  5:57       ` Minchan Kim
2014-01-23  5:57         ` Minchan Kim
  -- strict thread matches above, loose matches on Subject: below --
2013-12-02 19:21 [patch 0/9] mm: thrash detection-based file cache sizing v7 Johannes Weiner
2013-12-02 19:21 ` [patch 9/9] mm: keep page cache radix tree nodes in check Johannes Weiner
2013-12-02 19:21   ` Johannes Weiner
2013-12-02 22:10   ` Dave Chinner
2013-12-02 22:10     ` Dave Chinner
2013-12-02 22:46     ` Johannes Weiner
2013-12-02 22:46       ` Johannes Weiner
2013-11-24 23:38 [patch 0/9] mm: thrash detection-based file cache sizing v6 Johannes Weiner
2013-11-24 23:38 ` [patch 9/9] mm: keep page cache radix tree nodes in check Johannes Weiner
2013-11-24 23:38   ` Johannes Weiner
2013-11-25 23:49   ` Dave Chinner
2013-11-25 23:49     ` Dave Chinner
2013-11-26 21:27     ` Johannes Weiner
2013-11-26 21:27       ` Johannes Weiner
2013-11-26 22:29       ` Dave Chinner
2013-11-26 22:29         ` Dave Chinner
2013-11-26 23:00         ` Johannes Weiner
2013-11-26 23:00           ` Johannes Weiner
2013-11-27  0:59           ` Dave Chinner
2013-11-27  0:59             ` Dave Chinner
2013-11-26  0:13   ` Andrew Morton
2013-11-26  0:13     ` Andrew Morton
2013-11-26 22:05     ` Johannes Weiner
2013-11-26 22:05       ` Johannes Weiner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=52D622B5.6070203@oracle.com \
    --to=bob.liu@oracle.com \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=andi@firstfloor.org \
    --cc=david@fromorbit.com \
    --cc=gthelen@google.com \
    --cc=hannes@cmpxchg.org \
    --cc=hch@infradead.org \
    --cc=hughd@google.com \
    --cc=jack@suse.cz \
    --cc=klamm@yandex-team.ru \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=linux-mm@kvack.org \
    --cc=metin@citusdata.com \
    --cc=mgorman@suse.de \
    --cc=minchan.kim@gmail.com \
    --cc=ozgun@citusdata.com \
    --cc=peterz@infradead.org \
    --cc=riel@redhat.com \
    --cc=rmallon@gmail.com \
    --cc=semenzato@google.com \
    --cc=tj@kernel.org \
    --cc=vbabka@suse.cz \
    --cc=walken@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.