From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from zps37.corp.google.com (zps37.corp.google.com [172.25.146.37]) by smtp-out.google.com with ESMTP id lAG4Pj7m011508 for ; Thu, 15 Nov 2007 20:25:45 -0800 Received: from [172.18.116.11] (freakapc.corp.google.com [172.18.116.11]) by zps37.corp.google.com with ESMTP id lAG4PjRV026573 for ; Thu, 15 Nov 2007 20:25:45 -0800 Message-ID: <473D1BC9.8050904@google.com> Date: Thu, 15 Nov 2007 20:25:45 -0800 From: Ethan Solomita MIME-Version: 1.0 Subject: page_referenced() and VM_LOCKED Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org Return-Path: To: linux-mm@kvack.org List-ID: page_referenced_file() checks for the vma to be VM_LOCKED|VM_MAYSHARE and adds returns 1. We don't do the same in page_referenced_anon(). I would've thought the point was to treat locked pages as active, never pushing them into the inactive list, but since that's not quite what's happening I was hoping someone could give me a clue. Thanks, -- Ethan -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Date: Fri, 16 Nov 2007 14:46:41 +0900 From: KAMEZAWA Hiroyuki Subject: Re: page_referenced() and VM_LOCKED Message-Id: <20071116144641.f12fd610.kamezawa.hiroyu@jp.fujitsu.com> In-Reply-To: <473D1BC9.8050904@google.com> References: <473D1BC9.8050904@google.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org Return-Path: To: Ethan Solomita Cc: linux-mm@kvack.org List-ID: On Thu, 15 Nov 2007 20:25:45 -0800 Ethan Solomita wrote: > page_referenced_file() checks for the vma to be VM_LOCKED|VM_MAYSHARE > and adds returns 1. We don't do the same in page_referenced_anon(). I > would've thought the point was to treat locked pages as active, never > pushing them into the inactive list, but since that's not quite what's > happening I was hoping someone could give me a clue. > > Thanks, > -- Ethan Hmm, == vmscan.c::shrink_page_list() page_referenced() if returns 1 -> link to active list add to swap # only works if anon try_to_unmap() if VM_LOCKED -> SWAP_FAIL -> link to active list == Then, "VM_LOCKED & not referenced" anon page is added to swap cache (before pushed back to active list) Seems intended ? Thanks, - Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from l-036148a.enterprise.veritas.com([10.10.97.179]) (3025 bytes) by megami.veritas.com via sendmail with P:esmtp/R:smart_host/T:smtp (sender: ) id for ; Fri, 16 Nov 2007 10:00:25 -0800 (PST) (Smail-3.2.0.101 1997-Dec-17 #15 built 2001-Aug-30) Date: Fri, 16 Nov 2007 18:00:02 +0000 (GMT) From: Hugh Dickins Subject: Re: page_referenced() and VM_LOCKED In-Reply-To: <20071116144641.f12fd610.kamezawa.hiroyu@jp.fujitsu.com> Message-ID: References: <473D1BC9.8050904@google.com> <20071116144641.f12fd610.kamezawa.hiroyu@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org Return-Path: To: KAMEZAWA Hiroyuki Cc: Ethan Solomita , linux-mm@kvack.org List-ID: On Fri, 16 Nov 2007, KAMEZAWA Hiroyuki wrote: > On Thu, 15 Nov 2007 20:25:45 -0800 > Ethan Solomita wrote: > > > page_referenced_file() checks for the vma to be VM_LOCKED|VM_MAYSHARE > > and adds returns 1. That's a case where it can deduce that the page is present and should be treated as referenced, without even examining the page tables. > > We don't do the same in page_referenced_anon(). It cannot make that same deduction in the page_referenced_anon() case (different vmas may well contain different COWs of some original page). Perhaps you're suggesting that page_referenced_one() ought to cover this case. Yes. I think we were coming at it from the 2.6.4 rmap.c in which VM_LOCKED was checked only in the try_to_unmap() case; but I was worried about the length of the vma lists and tried to short- circuit the full search in the one case I could; without thinking about doing the equivalent elsewhere for the other cases. > > I would've thought the point was to treat locked pages as active, never > > pushing them into the inactive list, but since that's not quite what's > > happening I was hoping someone could give me a clue. Rik and Lee and others have proposed that we keep VM_LOCKED pages off both active and inactive lists: that seems a better way forward. > > > > Thanks, > > -- Ethan > Hmm, > > == vmscan.c::shrink_page_list() > > page_referenced() if returns 1 -> link to active list > > add to swap # only works if anon > > try_to_unmap() if VM_LOCKED -> SWAP_FAIL -> link to active list > > == > > Then, "VM_LOCKED & not referenced" anon page is added to swap cache > (before pushed back to active list) > > Seems intended ? Not intended, no. Rather a waste of swap. How about this patch? --- 2.6.24-rc2/mm/rmap.c 2007-10-24 07:16:04.000000000 +0100 +++ linux/mm/rmap.c 2007-11-16 17:45:32.000000000 +0000 @@ -283,7 +283,10 @@ static int page_referenced_one(struct pa if (!pte) goto out; - if (ptep_clear_flush_young(vma, address, pte)) + if (vma->vm_flags & VM_LOCKED) { + referenced++; + *mapcount = 1; + } else if (ptep_clear_flush_young(vma, address, pte)) referenced++; /* Pretend the page is referenced if the task has the -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: kamezawa.hiroyu@jp.fujitsu.com Message-ID: <16909246.1195259556869.kamezawa.hiroyu@jp.fujitsu.com> Date: Sat, 17 Nov 2007 09:32:36 +0900 (JST) Subject: Re: Re: page_referenced() and VM_LOCKED In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit References: <473D1BC9.8050904@google.com> <20071116144641.f12fd610.kamezawa.hiroyu@jp.fujitsu.com> Sender: owner-linux-mm@kvack.org Return-Path: To: Hugh Dickins Cc: KAMEZAWA Hiroyuki , Ethan Solomita , linux-mm@kvack.org List-ID: >> > I would've thought the point was to treat locked pages as active, never >> > pushing them into the inactive list, but since that's not quite what's >> > happening I was hoping someone could give me a clue. > >Rik and Lee and others have proposed that we keep VM_LOCKED pages >off both active and inactive lists: that seems a better way forward. > agreed. >> Then, "VM_LOCKED & not referenced" anon page is added to swap cache >> (before pushed back to active list) >> >> Seems intended ? > >Not intended, no. Rather a waste of swap. How about this patch? > seems nice. I'd like to do some test in the next week, Thanks, -Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Date: Mon, 19 Nov 2007 18:39:42 +0900 From: KAMEZAWA Hiroyuki Subject: Re: page_referenced() and VM_LOCKED Message-Id: <20071119183942.614771c2.kamezawa.hiroyu@jp.fujitsu.com> In-Reply-To: <16909246.1195259556869.kamezawa.hiroyu@jp.fujitsu.com> References: <473D1BC9.8050904@google.com> <20071116144641.f12fd610.kamezawa.hiroyu@jp.fujitsu.com> <16909246.1195259556869.kamezawa.hiroyu@jp.fujitsu.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org Return-Path: To: kamezawa.hiroyu@jp.fujitsu.com Cc: Hugh Dickins , Ethan Solomita , linux-mm@kvack.org List-ID: On Sat, 17 Nov 2007 09:32:36 +0900 (JST) kamezawa.hiroyu@jp.fujitsu.com wrote: > >> > I would've thought the point was to treat locked pages as active, never > >> > pushing them into the inactive list, but since that's not quite what's > >> > happening I was hoping someone could give me a clue. > > > >Rik and Lee and others have proposed that we keep VM_LOCKED pages > >off both active and inactive lists: that seems a better way forward. > > > agreed. > > >> Then, "VM_LOCKED & not referenced" anon page is added to swap cache > >> (before pushed back to active list) > >> > >> Seems intended ? > > > >Not intended, no. Rather a waste of swap. How about this patch? > > > seems nice. I'd like to do some test in the next week, > your patch helps the kernel to avoid a waste of Swap. Tested-by: KAMEZAWA Hiroyuki == I tested your patch on x86_64/6GiB memory, + 2.6.24-rc3. mlock 5GiB and create 4GiB file by"dd". [before patch] MemTotal: 6072620 kB MemFree: 50540 kB Buffers: 4508 kB Cached: 724828 kB SwapCached: 5146960 kB Active: 2683964 kB Inactive: 3198752 kB [after patch] MemTotal: 6072620 kB MemFree: 17112 kB Buffers: 6816 kB Cached: 744880 kB SwapCached: 21724 kB Active: 5175828 kB Inactive: 744956 kB Thanks, -Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from l-036148a.enterprise.veritas.com([10.10.97.179]) (1751 bytes) by megami.veritas.com via sendmail with P:esmtp/R:smart_host/T:smtp (sender: ) id for ; Mon, 19 Nov 2007 11:09:43 -0800 (PST) (Smail-3.2.0.101 1997-Dec-17 #15 built 2001-Aug-30) Date: Mon, 19 Nov 2007 19:09:25 +0000 (GMT) From: Hugh Dickins Subject: Re: page_referenced() and VM_LOCKED In-Reply-To: <20071119183942.614771c2.kamezawa.hiroyu@jp.fujitsu.com> Message-ID: References: <473D1BC9.8050904@google.com> <20071116144641.f12fd610.kamezawa.hiroyu@jp.fujitsu.com> <16909246.1195259556869.kamezawa.hiroyu@jp.fujitsu.com> <20071119183942.614771c2.kamezawa.hiroyu@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org Return-Path: To: KAMEZAWA Hiroyuki Cc: Ethan Solomita , linux-mm@kvack.org List-ID: On Mon, 19 Nov 2007, KAMEZAWA Hiroyuki wrote: > > > your patch helps the kernel to avoid a waste of Swap. > > Tested-by: KAMEZAWA Hiroyuki Many thanks, Kame. I'll send it in with my next little lot. Hugh > == > I tested your patch on x86_64/6GiB memory, + 2.6.24-rc3. > mlock 5GiB and create 4GiB file by"dd". > > [before patch] > MemTotal: 6072620 kB > MemFree: 50540 kB > Buffers: 4508 kB > Cached: 724828 kB > SwapCached: 5146960 kB > Active: 2683964 kB > Inactive: 3198752 kB > > [after patch] > MemTotal: 6072620 kB > MemFree: 17112 kB > Buffers: 6816 kB > Cached: 744880 kB > SwapCached: 21724 kB > Active: 5175828 kB > Inactive: 744956 kB -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4755F041.4000605@google.com> Date: Tue, 04 Dec 2007 16:26:41 -0800 From: Ethan Solomita MIME-Version: 1.0 Subject: Re: page_referenced() and VM_LOCKED References: <473D1BC9.8050904@google.com> <20071116144641.f12fd610.kamezawa.hiroyu@jp.fujitsu.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org Return-Path: To: Hugh Dickins Cc: KAMEZAWA Hiroyuki , linux-mm@kvack.org List-ID: Hugh Dickins wrote: > On Fri, 16 Nov 2007, KAMEZAWA Hiroyuki wrote: >> On Thu, 15 Nov 2007 20:25:45 -0800 >> Ethan Solomita wrote: >> >>> page_referenced_file() checks for the vma to be VM_LOCKED|VM_MAYSHARE >>> and adds returns 1. > > That's a case where it can deduce that the page is present and should > be treated as referenced, without even examining the page tables. > >>> We don't do the same in page_referenced_anon(). > > It cannot make that same deduction in the page_referenced_anon() case > (different vmas may well contain different COWs of some original page). Sorry to come back in with this so late -- if the vma is VM_MAYSHARE, would there be COWs of the original page? -- Ethan -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [10.10.97.164]([10.10.97.164]) (2361 bytes) by megami.veritas.com via sendmail with P:esmtp/R:smart_host/T:smtp (sender: ) id for ; Wed, 5 Dec 2007 07:55:12 -0800 (PST) (Smail-3.2.0.101 1997-Dec-17 #15 built 2001-Aug-30) Date: Wed, 5 Dec 2007 15:54:46 +0000 (GMT) From: Hugh Dickins Subject: Re: page_referenced() and VM_LOCKED In-Reply-To: <4755F041.4000605@google.com> Message-ID: References: <473D1BC9.8050904@google.com> <20071116144641.f12fd610.kamezawa.hiroyu@jp.fujitsu.com> <4755F041.4000605@google.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org Return-Path: To: Ethan Solomita Cc: KAMEZAWA Hiroyuki , linux-mm@kvack.org List-ID: On Tue, 4 Dec 2007, Ethan Solomita wrote: > Hugh Dickins wrote: > > On Fri, 16 Nov 2007, KAMEZAWA Hiroyuki wrote: > > > On Thu, 15 Nov 2007 20:25:45 -0800 > > > Ethan Solomita wrote: > > > > > > > page_referenced_file() checks for the vma to be VM_LOCKED|VM_MAYSHARE > > > > and adds returns 1. > > > > That's a case where it can deduce that the page is present and should > > be treated as referenced, without even examining the page tables. > > > > > > We don't do the same in page_referenced_anon(). > > > > It cannot make that same deduction in the page_referenced_anon() case > > (different vmas may well contain different COWs of some original page). > > Sorry to come back in with this so late -- > if the vma is VM_MAYSHARE, would there be COWs of the original page? 99.999% correct answer: No, that's why we're testing VM_MAYSHARE, because we know it has to be that original page which is present and locked there (it might be readwrite, or it might be readonly, but it won't be a COWed copy in a VM_MAYSHARE vma; the same would be true if we tested VM_SHARED, but that would miss readonly cases). 00.001% adjustment: Actually, there's an aberrant case in which do_wp_page() can put a COW into a VM_MAYSHARE area, supposedly to suit ptrace(). I overlooked that case when I put the VM_MAYSHARE test into page_referenced_file(); later when I learnt about it (and found Linus unwilling to change it), I did an audit of such oversights, but concluded this test wasn't worth changing. Hugh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org