All of lore.kernel.org
 help / color / mirror / Atom feed
From: Avi Kivity <avi@redhat.com>
To: Wu Fengguang <fengguang.wu@intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>,
	Rik van Riel <riel@redhat.com>,
	"Dike, Jeffrey G" <jeffrey.g.dike@intel.com>,
	"Yu, Wilfred" <wilfred.yu@intel.com>,
	"Kleen, Andi" <andi.kleen@intel.com>,
	Hugh Dickins <hugh.dickins@tiscali.co.uk>,
	Andrew Morton <akpm@linux-foundation.org>,
	Christoph Lameter <cl@linux-foundation.org>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Mel Gorman <mel@csn.ul.ie>, LKML <linux-kernel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>
Subject: Re: [RFC] respect the referenced bit of KVM guest pages?
Date: Thu, 06 Aug 2009 16:46:57 +0300	[thread overview]
Message-ID: <4A7ADED1.5030408@redhat.com> (raw)
In-Reply-To: <20090806130631.GB6162@localhost>

On 08/06/2009 04:06 PM, Wu Fengguang wrote:
> On Thu, Aug 06, 2009 at 07:44:01PM +0800, Avi Kivity wrote:
>    
>> On 08/06/2009 01:59 PM, Wu Fengguang wrote:
>>      
>
> scheme KEEP_MOST:
>
>    
>>> How about, for every N pages that you scan, evict at least 1 page,
>>> regardless of young bit status?  That limits overscanning to a N:1
>>> ratio.  With N=250 we'll spend at most 25 usec in order to locate one
>>> page to evict.
>>>        
>
> scheme DROP_CONTINUOUS:
>
>    
>>> This is a quick hack to materialize the idea. It remembers roughly
>>> the last 32*SWAP_CLUSTER_MAX=1024 active (mapped) pages scanned,
>>> and if _all of them_ are referenced, then the referenced bit is
>>> probably meaningless and should not be taken seriously.
>>>        
>
>    

Or one scheme, with N=parameter.

>> I don't think we should ignore the referenced bit. There could still be
>> a large batch of unreferenced pages later on that we should
>> preferentially swap. If we swap at least 1 page for every 250 scanned,
>> after 4K swaps we will have traversed 1M pages, enough to find them.
>>      
>
> I guess both schemes have unacceptable flaws.
>
> For JVM/BIGMEM workload, most pages would be found referenced _all the time_.
> So the KEEP_MOST scheme could increase reclaim overheads by N=250 times;
> while the DROP_CONTINUOUS scheme is effectively zero cost.
>    

Maybe 250 is an exaggeration.  But even with 250, the cost is still 
pretty low compared to the cpu cost of evicting a page (with IPIs and 
tlb flushes).

-- 
error compiling committee.c: too many arguments to function


WARNING: multiple messages have this Message-ID (diff)
From: Avi Kivity <avi@redhat.com>
To: Wu Fengguang <fengguang.wu@intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>,
	Rik van Riel <riel@redhat.com>,
	"Dike, Jeffrey G" <jeffrey.g.dike@intel.com>,
	"Yu, Wilfred" <wilfred.yu@intel.com>,
	"Kleen, Andi" <andi.kleen@intel.com>,
	Hugh Dickins <hugh.dickins@tiscali.co.uk>,
	Andrew Morton <akpm@linux-foundation.org>,
	Christoph Lameter <cl@linux-foundation.org>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Mel Gorman <mel@csn.ul.ie>, LKML <linux-kernel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>
Subject: Re: [RFC] respect the referenced bit of KVM guest pages?
Date: Thu, 06 Aug 2009 16:46:57 +0300	[thread overview]
Message-ID: <4A7ADED1.5030408@redhat.com> (raw)
In-Reply-To: <20090806130631.GB6162@localhost>

On 08/06/2009 04:06 PM, Wu Fengguang wrote:
> On Thu, Aug 06, 2009 at 07:44:01PM +0800, Avi Kivity wrote:
>    
>> On 08/06/2009 01:59 PM, Wu Fengguang wrote:
>>      
>
> scheme KEEP_MOST:
>
>    
>>> How about, for every N pages that you scan, evict at least 1 page,
>>> regardless of young bit status?  That limits overscanning to a N:1
>>> ratio.  With N=250 we'll spend at most 25 usec in order to locate one
>>> page to evict.
>>>        
>
> scheme DROP_CONTINUOUS:
>
>    
>>> This is a quick hack to materialize the idea. It remembers roughly
>>> the last 32*SWAP_CLUSTER_MAX=1024 active (mapped) pages scanned,
>>> and if _all of them_ are referenced, then the referenced bit is
>>> probably meaningless and should not be taken seriously.
>>>        
>
>    

Or one scheme, with N=parameter.

>> I don't think we should ignore the referenced bit. There could still be
>> a large batch of unreferenced pages later on that we should
>> preferentially swap. If we swap at least 1 page for every 250 scanned,
>> after 4K swaps we will have traversed 1M pages, enough to find them.
>>      
>
> I guess both schemes have unacceptable flaws.
>
> For JVM/BIGMEM workload, most pages would be found referenced _all the time_.
> So the KEEP_MOST scheme could increase reclaim overheads by N=250 times;
> while the DROP_CONTINUOUS scheme is effectively zero cost.
>    

Maybe 250 is an exaggeration.  But even with 250, the cost is still 
pretty low compared to the cpu cost of evicting a page (with IPIs and 
tlb flushes).

-- 
error compiling committee.c: too many arguments to function

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2009-08-06 13:42 UTC|newest]

Thread overview: 243+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-08-05  2:40 [RFC] respect the referenced bit of KVM guest pages? Wu Fengguang
2009-08-05  2:40 ` Wu Fengguang
2009-08-05  4:15 ` KOSAKI Motohiro
2009-08-05  4:15   ` KOSAKI Motohiro
2009-08-05  4:41   ` Wu Fengguang
2009-08-05  4:41     ` Wu Fengguang
2009-08-05  7:58 ` Avi Kivity
2009-08-05  7:58   ` Avi Kivity
2009-08-05  8:17   ` Avi Kivity
2009-08-05  8:17     ` Avi Kivity
2009-08-05 14:33     ` Rik van Riel
2009-08-05 14:33       ` Rik van Riel
2009-08-05 15:37       ` Avi Kivity
2009-08-05 15:37         ` Avi Kivity
2009-08-05 14:15   ` Rik van Riel
2009-08-05 14:15     ` Rik van Riel
2009-08-05 15:12     ` Avi Kivity
2009-08-05 15:12       ` Avi Kivity
2009-08-05 15:15       ` Rik van Riel
2009-08-05 15:15         ` Rik van Riel
2009-08-05 15:25         ` Avi Kivity
2009-08-05 15:25           ` Avi Kivity
2009-08-05 16:35           ` Andrea Arcangeli
2009-08-05 16:35             ` Andrea Arcangeli
2009-08-05 16:31     ` Andrea Arcangeli
2009-08-05 16:31       ` Andrea Arcangeli
2009-08-05 17:25       ` Rik van Riel
2009-08-05 17:25         ` Rik van Riel
2009-08-05 15:45   ` Dike, Jeffrey G
2009-08-05 15:45     ` Dike, Jeffrey G
2009-08-05 16:05   ` Andrea Arcangeli
2009-08-05 16:05     ` Andrea Arcangeli
2009-08-05 16:12     ` Dike, Jeffrey G
2009-08-05 16:12       ` Dike, Jeffrey G
2009-08-05 16:19       ` Andrea Arcangeli
2009-08-05 16:19         ` Andrea Arcangeli
2009-08-05 15:58 ` Andrea Arcangeli
2009-08-05 15:58   ` Andrea Arcangeli
2009-08-05 17:20   ` Rik van Riel
2009-08-05 17:20     ` Rik van Riel
2009-08-05 17:42   ` Rik van Riel
2009-08-05 17:42     ` Rik van Riel
2009-08-06 10:15     ` Andrea Arcangeli
2009-08-06 10:15       ` Andrea Arcangeli
2009-08-06 10:08   ` Andrea Arcangeli
2009-08-06 10:08     ` Andrea Arcangeli
2009-08-06 10:18     ` Avi Kivity
2009-08-06 10:18       ` Avi Kivity
2009-08-06 10:20       ` Andrea Arcangeli
2009-08-06 10:20         ` Andrea Arcangeli
2009-08-06 10:59         ` Wu Fengguang
2009-08-06 10:59           ` Wu Fengguang
2009-08-06 11:44           ` Avi Kivity
2009-08-06 11:44             ` Avi Kivity
2009-08-06 13:06             ` Wu Fengguang
2009-08-06 13:06               ` Wu Fengguang
2009-08-06 13:16               ` Rik van Riel
2009-08-06 13:16                 ` Rik van Riel
2009-08-16  3:28                 ` Wu Fengguang
2009-08-16  3:28                   ` Wu Fengguang
2009-08-16  3:56                   ` Rik van Riel
2009-08-16  3:56                     ` Rik van Riel
2009-08-16  4:43                     ` Balbir Singh
2009-08-16  4:43                       ` Balbir Singh
2009-08-16  4:55                     ` Wu Fengguang
2009-08-16  4:55                       ` Wu Fengguang
2009-08-16  5:59                       ` Balbir Singh
2009-08-16  5:59                         ` Balbir Singh
2009-08-17 19:47                       ` Dike, Jeffrey G
2009-08-17 19:47                         ` Dike, Jeffrey G
2009-08-21 18:24                         ` Balbir Singh
2009-08-21 18:24                           ` Balbir Singh
2009-08-31 19:43                           ` Dike, Jeffrey G
2009-08-31 19:43                             ` Dike, Jeffrey G
2009-08-31 19:52                             ` Rik van Riel
2009-08-31 19:52                               ` Rik van Riel
2009-08-31 20:06                               ` Dike, Jeffrey G
2009-08-31 20:06                                 ` Dike, Jeffrey G
2009-08-31 20:09                                 ` Rik van Riel
2009-08-31 20:09                                   ` Rik van Riel
2009-08-31 20:11                                   ` Dike, Jeffrey G
2009-08-31 20:11                                     ` Dike, Jeffrey G
2009-08-31 20:42                                     ` Balbir Singh
2009-08-31 20:42                                       ` Balbir Singh
2009-08-06 13:46               ` Avi Kivity [this message]
2009-08-06 13:46                 ` Avi Kivity
2009-08-06 21:09               ` Jeff Dike
2009-08-06 21:09                 ` Jeff Dike
2009-08-16  3:18                 ` Wu Fengguang
2009-08-16  3:18                   ` Wu Fengguang
2009-08-16  3:53                   ` Rik van Riel
2009-08-16  3:53                     ` Rik van Riel
2009-08-16  5:15                     ` Wu Fengguang
2009-08-16  5:15                       ` Wu Fengguang
2009-08-16 11:29                       ` Wu Fengguang
2009-08-16 11:29                         ` Wu Fengguang
2009-08-17 14:33                         ` Minchan Kim
2009-08-17 14:33                           ` Minchan Kim
2009-08-18  2:34                           ` Wu Fengguang
2009-08-18  2:34                             ` Wu Fengguang
2009-08-18  4:17                             ` Minchan Kim
2009-08-18  4:17                               ` Minchan Kim
2009-08-18  9:31                               ` Wu Fengguang
2009-08-18  9:31                                 ` Wu Fengguang
2009-08-18  9:52                                 ` Minchan Kim
2009-08-18  9:52                                   ` Minchan Kim
2009-08-18 10:00                                   ` Wu Fengguang
2009-08-18 10:00                                     ` Wu Fengguang
2009-08-18 11:00                                     ` Minchan Kim
2009-08-18 11:00                                       ` Minchan Kim
2009-08-18 11:11                                       ` Wu Fengguang
2009-08-18 11:11                                         ` Wu Fengguang
2009-08-18 14:03                                         ` Minchan Kim
2009-08-18 14:03                                           ` Minchan Kim
2009-08-18 16:27                                         ` KOSAKI Motohiro
2009-08-18 16:27                                           ` KOSAKI Motohiro
2009-08-18 15:57                         ` KOSAKI Motohiro
2009-08-18 15:57                           ` KOSAKI Motohiro
2009-08-19 12:01                           ` Wu Fengguang
2009-08-19 12:01                             ` Wu Fengguang
2009-08-19 12:05                             ` KOSAKI Motohiro
2009-08-19 12:05                               ` KOSAKI Motohiro
2009-08-19 12:10                               ` Wu Fengguang
2009-08-19 12:10                                 ` Wu Fengguang
2009-08-19 12:25                                 ` Minchan Kim
2009-08-19 12:25                                   ` Minchan Kim
2009-08-19 13:19                                   ` KOSAKI Motohiro
2009-08-19 13:19                                     ` KOSAKI Motohiro
2009-08-19 13:28                                     ` Minchan Kim
2009-08-19 13:28                                       ` Minchan Kim
2009-08-21 11:17                                       ` KOSAKI Motohiro
2009-08-21 11:17                                         ` KOSAKI Motohiro
2009-08-19 13:24                                   ` Wu Fengguang
2009-08-19 13:24                                     ` Wu Fengguang
2009-08-19 13:38                                     ` Minchan Kim
2009-08-19 13:38                                       ` Minchan Kim
2009-08-19 14:00                                       ` Wu Fengguang
2009-08-19 14:00                                         ` Wu Fengguang
2009-08-06 13:13             ` Rik van Riel
2009-08-06 13:13               ` Rik van Riel
2009-08-06 13:49               ` Avi Kivity
2009-08-06 13:49                 ` Avi Kivity
2009-08-07  3:11               ` KOSAKI Motohiro
2009-08-07  3:11                 ` KOSAKI Motohiro
2009-08-07  7:54                 ` Balbir Singh
2009-08-07  7:54                   ` Balbir Singh
2009-08-07  8:24                   ` KAMEZAWA Hiroyuki
2009-08-07  8:24                     ` KAMEZAWA Hiroyuki
2009-08-06 13:11           ` Rik van Riel
2009-08-06 13:11             ` Rik van Riel
2009-08-06 13:08     ` Rik van Riel
2009-08-06 13:08       ` Rik van Riel
2009-08-07  3:17       ` KOSAKI Motohiro
2009-08-07  3:17         ` KOSAKI Motohiro
2009-08-12  7:48         ` Wu Fengguang
2009-08-12 14:31           ` Rik van Riel
2009-08-12 14:31             ` Rik van Riel
2009-08-13  1:03             ` Wu Fengguang
2009-08-13  1:03               ` Wu Fengguang
2009-08-13 15:46               ` Rik van Riel
2009-08-13 15:46                 ` Rik van Riel
2009-08-13 16:12                 ` Avi Kivity
2009-08-13 16:12                   ` Avi Kivity
2009-08-13 16:26                   ` Rik van Riel
2009-08-13 16:26                     ` Rik van Riel
2009-08-13 19:12                     ` Avi Kivity
2009-08-13 19:12                       ` Avi Kivity
2009-08-13 21:16                       ` Johannes Weiner
2009-08-13 21:16                         ` Johannes Weiner
2009-08-14  7:16                         ` Avi Kivity
2009-08-14  7:16                           ` Avi Kivity
2009-08-14  9:10                           ` Johannes Weiner
2009-08-14  9:10                             ` Johannes Weiner
2009-08-14  9:51                             ` Wu Fengguang
2009-08-14  9:51                               ` Wu Fengguang
2009-08-14 13:19                               ` Rik van Riel
2009-08-14 13:19                                 ` Rik van Riel
2009-08-15  5:45                                 ` Wu Fengguang
2009-08-15  5:45                                   ` Wu Fengguang
2009-08-16  5:09                                   ` Balbir Singh
2009-08-16  5:09                                     ` Balbir Singh
2009-08-16  5:41                                     ` Wu Fengguang
2009-08-16  5:41                                       ` Wu Fengguang
2009-08-16  5:50                                     ` Wu Fengguang
2009-08-16  5:50                                       ` Wu Fengguang
2009-08-18 15:57                                     ` KOSAKI Motohiro
2009-08-18 15:57                                       ` KOSAKI Motohiro
2009-08-17 18:04                                   ` Dike, Jeffrey G
2009-08-17 18:04                                     ` Dike, Jeffrey G
2009-08-18  2:26                                     ` Wu Fengguang
2009-08-18  2:26                                       ` Wu Fengguang
2009-09-02 19:30                                       ` Dike, Jeffrey G
2009-09-02 19:30                                         ` Dike, Jeffrey G
2009-09-03  2:04                                         ` Wu Fengguang
2009-09-03  2:04                                           ` Wu Fengguang
2009-09-04 20:06                                           ` Dike, Jeffrey G
2009-09-04 20:06                                             ` Dike, Jeffrey G
2009-09-04 20:57                                             ` Rik van Riel
2009-09-04 20:57                                               ` Rik van Riel
2009-08-18 15:57                                   ` KOSAKI Motohiro
2009-08-18 15:57                                     ` KOSAKI Motohiro
2009-08-19 12:08                                     ` Wu Fengguang
2009-08-19 12:08                                       ` Wu Fengguang
2009-08-19 13:40                                     ` [RFC] memcg: move definitions to .h and inline some functions Wu Fengguang
2009-08-19 13:40                                       ` Wu Fengguang
2009-08-19 14:18                                       ` KAMEZAWA Hiroyuki
2009-08-19 14:18                                         ` KAMEZAWA Hiroyuki
2009-08-19 14:27                                         ` Balbir Singh
2009-08-19 14:27                                           ` Balbir Singh
2009-08-20  1:34                                           ` Wu Fengguang
2009-08-20  1:34                                             ` Wu Fengguang
2009-08-14 21:42                               ` [RFC] respect the referenced bit of KVM guest pages? Dike, Jeffrey G
2009-08-14 21:42                                 ` Dike, Jeffrey G
2009-08-14 22:37                                 ` Rik van Riel
2009-08-14 22:37                                   ` Rik van Riel
2009-08-15  5:32                                   ` Wu Fengguang
2009-08-15  5:32                                     ` Wu Fengguang
2009-09-13 16:23                                 ` KOSAKI Motohiro
2009-09-13 16:23                                   ` KOSAKI Motohiro
2009-08-05 17:53 ` Rik van Riel
2009-08-05 17:53   ` Rik van Riel
2009-08-05 19:00   ` Dike, Jeffrey G
2009-08-05 19:00     ` Dike, Jeffrey G
2009-08-05 19:07     ` Rik van Riel
2009-08-05 19:07       ` Rik van Riel
2009-08-05 19:18       ` Dike, Jeffrey G
2009-08-05 19:18         ` Dike, Jeffrey G
2009-08-06  9:22         ` Avi Kivity
2009-08-06  9:22           ` Avi Kivity
2009-08-06  9:25           ` Wu Fengguang
2009-08-06  9:25             ` Wu Fengguang
2009-08-06  9:35             ` Avi Kivity
2009-08-06  9:35               ` Avi Kivity
2009-08-06  9:35               ` Wu Fengguang
2009-08-06  9:35                 ` Wu Fengguang
2009-08-06  9:59                 ` Avi Kivity
2009-08-06  9:59                   ` Avi Kivity
2009-08-06  9:59                   ` Wu Fengguang
2009-08-06  9:59                     ` Wu Fengguang
2009-08-06 10:14                     ` Avi Kivity
2009-08-06 10:14                       ` Avi Kivity
2009-08-07  1:25                       ` KAMEZAWA Hiroyuki
2009-08-07  1:25                         ` KAMEZAWA Hiroyuki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4A7ADED1.5030408@redhat.com \
    --to=avi@redhat.com \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=andi.kleen@intel.com \
    --cc=cl@linux-foundation.org \
    --cc=fengguang.wu@intel.com \
    --cc=hugh.dickins@tiscali.co.uk \
    --cc=jeffrey.g.dike@intel.com \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    --cc=riel@redhat.com \
    --cc=wilfred.yu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.