From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx147.postini.com [74.125.245.147]) by kanga.kvack.org (Postfix) with SMTP id 647936B0034 for ; Tue, 23 Apr 2013 08:08:22 -0400 (EDT) Received: by mail-pd0-f178.google.com with SMTP id w11so395767pde.37 for ; Tue, 23 Apr 2013 05:08:21 -0700 (PDT) Date: Tue, 23 Apr 2013 20:25:42 +0800 From: Zheng Liu Subject: [question] call mark_page_accessed() in minor fault Message-ID: <20130423122542.GA5638@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-linux-mm@kvack.org List-ID: To: linux-mm@kvack.org Cc: muming.wq@taobao.com Hi all, Recently we meet a performance regression about mmaped page. When we upgrade our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel, we will find that mmaped pages are reclaimed very quickly. We found that when we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in 2.6.32 kernel we don't call mark_page_accesed(). This means that mmaped pages in 2.6.18 kernel are activated and moved into active list. While in 2.6.32 kernel mmaped pages are still kept in inactive list. So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't call it in 2.6.32 kernel. Has any reason here? Thanks in advance, - Zheng -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx104.postini.com [74.125.245.104]) by kanga.kvack.org (Postfix) with SMTP id 774346B0033 for ; Tue, 23 Apr 2013 09:02:39 -0400 (EDT) Received: by mail-lb0-f180.google.com with SMTP id t11so621629lbi.11 for ; Tue, 23 Apr 2013 06:02:37 -0700 (PDT) Message-ID: <5176866A.2060400@openvz.org> Date: Tue, 23 Apr 2013 17:02:34 +0400 From: Konstantin Khlebnikov MIME-Version: 1.0 Subject: Re: [question] call mark_page_accessed() in minor fault References: <20130423122542.GA5638@gmail.com> In-Reply-To: <20130423122542.GA5638@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Zheng Liu Cc: linux-mm@kvack.org, muming.wq@taobao.com Zheng Liu wrote: > Hi all, > > Recently we meet a performance regression about mmaped page. When we upgrade > our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel, > we will find that mmaped pages are reclaimed very quickly. We found that when > we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in > 2.6.32 kernel we don't call mark_page_accesed(). This means that mmaped pages > in 2.6.18 kernel are activated and moved into active list. While in 2.6.32 > kernel mmaped pages are still kept in inactive list. > > So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't > call it in 2.6.32 kernel. Has any reason here? Behavior was changed in commit v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path" Please see also commits v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and v3.2-4877-gc909e99 "vmscan: activate executable pages after first usage". Probably they can solve some of your problems. > > Thanks in advance, > - Zheng > > -- > To unsubscribe, send a message with 'unsubscribe linux-mm' in > the body to majordomo@kvack.org. For more info on Linux MM, > see: http://www.linux-mm.org/ . > Don't email: email@kvack.org -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx176.postini.com [74.125.245.176]) by kanga.kvack.org (Postfix) with SMTP id D2F126B0033 for ; Tue, 23 Apr 2013 09:32:13 -0400 (EDT) Received: by mail-da0-f47.google.com with SMTP id p1so344456dad.20 for ; Tue, 23 Apr 2013 06:32:13 -0700 (PDT) Date: Tue, 23 Apr 2013 21:49:35 +0800 From: Zheng Liu Subject: Re: [question] call mark_page_accessed() in minor fault Message-ID: <20130423134935.GA10138@gmail.com> References: <20130423122542.GA5638@gmail.com> <5176866A.2060400@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5176866A.2060400@openvz.org> Sender: owner-linux-mm@kvack.org List-ID: To: Konstantin Khlebnikov Cc: linux-mm@kvack.org, muming.wq@taobao.com Hi Konstantin, On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote: > Zheng Liu wrote: > >Hi all, > > > >Recently we meet a performance regression about mmaped page. When we upgrade > >our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel, > >we will find that mmaped pages are reclaimed very quickly. We found that when > >we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in > >2.6.32 kernel we don't call mark_page_accesed(). This means that mmaped pages > >in 2.6.18 kernel are activated and moved into active list. While in 2.6.32 > >kernel mmaped pages are still kept in inactive list. > > > >So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't > >call it in 2.6.32 kernel. Has any reason here? > > Behavior was changed in commit > v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path" Thanks for pointing it out. > > Please see also commits > v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and Yes, I will give it try. If I understand correctly, this commit is useful for multi-processes program that access a shared mmaped page, but that could not be useful for us because our program is multi-thread. > v3.2-4877-gc909e99 "vmscan: activate executable pages after first usage". We have backported this patch, but it is useless. This commit only tries to activate a executable page, but our mmaped pages aren't with this flag. Additional question is that currently mmaped page is reclaimed too quickly. I think maybe we need to adjust our page reclaim strategy to balance number of pages between mmaped page and file page cache. Now every time we access a page using read(2)/write(2), this page will be touched. But after first time we touch a mmaped page, we never touch it again except that this page is evicted. Regards, - Zheng -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx133.postini.com [74.125.245.133]) by kanga.kvack.org (Postfix) with SMTP id 2E19B6B0034 for ; Sat, 27 Apr 2013 03:10:38 -0400 (EDT) Received: by mail-pd0-f174.google.com with SMTP id y13so2425057pdi.33 for ; Sat, 27 Apr 2013 00:10:37 -0700 (PDT) Message-ID: <517B79E6.5050204@gmail.com> Date: Sat, 27 Apr 2013 15:10:30 +0800 From: Simon Jeons MIME-Version: 1.0 Subject: Re: [question] call mark_page_accessed() in minor fault References: <20130423122542.GA5638@gmail.com> <5176866A.2060400@openvz.org> <20130423134935.GA10138@gmail.com> In-Reply-To: <20130423134935.GA10138@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Zheng Liu Cc: Konstantin Khlebnikov , linux-mm@kvack.org, muming.wq@taobao.com Hi Zheng, On 04/23/2013 09:49 PM, Zheng Liu wrote: > Hi Konstantin, > > On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote: >> Zheng Liu wrote: >>> Hi all, >>> >>> Recently we meet a performance regression about mmaped page. When we upgrade >>> our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel, >>> we will find that mmaped pages are reclaimed very quickly. We found that when >>> we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in >>> 2.6.32 kernel we don't call mark_page_accesed(). This means that mmaped pages >>> in 2.6.18 kernel are activated and moved into active list. While in 2.6.32 >>> kernel mmaped pages are still kept in inactive list. >>> >>> So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't >>> call it in 2.6.32 kernel. Has any reason here? >> Behavior was changed in commit >> v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path" > Thanks for pointing it out. > >> Please see also commits >> v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and > Yes, I will give it try. If I understand correctly, this commit is > useful for multi-processes program that access a shared mmaped page, > but that could not be useful for us because our program is multi-thread. What's the difference behavior between multi-processes and multi-thread in this case? > >> v3.2-4877-gc909e99 "vmscan: activate executable pages after first usage". > We have backported this patch, but it is useless. This commit only > tries to activate a executable page, but our mmaped pages aren't with > this flag. > > Additional question is that currently mmaped page is reclaimed too > quickly. I think maybe we need to adjust our page reclaim strategy to > balance number of pages between mmaped page and file page cache. Now > every time we access a page using read(2)/write(2), this page will be > touched. But after first time we touch a mmaped page, we never touch it > again except that this page is evicted. > > Regards, > - Zheng > > -- > To unsubscribe, send a message with 'unsubscribe linux-mm' in > the body to majordomo@kvack.org. For more info on Linux MM, > see: http://www.linux-mm.org/ . > Don't email: email@kvack.org -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx170.postini.com [74.125.245.170]) by kanga.kvack.org (Postfix) with SMTP id B19386B0032 for ; Sat, 27 Apr 2013 03:37:51 -0400 (EDT) Received: by mail-pd0-f171.google.com with SMTP id t10so20560pdi.16 for ; Sat, 27 Apr 2013 00:37:51 -0700 (PDT) Date: Sat, 27 Apr 2013 15:55:16 +0800 From: Zheng Liu Subject: Re: [question] call mark_page_accessed() in minor fault Message-ID: <20130427075516.GA31442@gmail.com> References: <20130423122542.GA5638@gmail.com> <5176866A.2060400@openvz.org> <20130423134935.GA10138@gmail.com> <517B79E6.5050204@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <517B79E6.5050204@gmail.com> Sender: owner-linux-mm@kvack.org List-ID: To: Simon Jeons Cc: Konstantin Khlebnikov , linux-mm@kvack.org, muming.wq@taobao.com On Sat, Apr 27, 2013 at 03:10:30PM +0800, Simon Jeons wrote: > Hi Zheng, > On 04/23/2013 09:49 PM, Zheng Liu wrote: > >Hi Konstantin, > > > >On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote: > >>Zheng Liu wrote: > >>>Hi all, > >>> > >>>Recently we meet a performance regression about mmaped page. When we upgrade > >>>our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel, > >>>we will find that mmaped pages are reclaimed very quickly. We found that when > >>>we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in > >>>2.6.32 kernel we don't call mark_page_accesed(). This means that mmaped pages > >>>in 2.6.18 kernel are activated and moved into active list. While in 2.6.32 > >>>kernel mmaped pages are still kept in inactive list. > >>> > >>>So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't > >>>call it in 2.6.32 kernel. Has any reason here? > >>Behavior was changed in commit > >>v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path" > >Thanks for pointing it out. > > > >>Please see also commits > >>v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and > >Yes, I will give it try. If I understand correctly, this commit is > >useful for multi-processes program that access a shared mmaped page, > >but that could not be useful for us because our program is multi-thread. > > What's the difference behavior between multi-processes and > multi-thread in this case? Hi Simon, Sorry, I am not a MM expert. IIUC, if we have two processes, this mmaped page will be moved into active list. But if we only have two threads, reference_ptes == 1, and this mmaped page won't be moved into active list. Finally this page could be evicted. Am I missing something? Thanks, - Zheng -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx166.postini.com [74.125.245.166]) by kanga.kvack.org (Postfix) with SMTP id B915B6B0032 for ; Sat, 27 Apr 2013 03:40:23 -0400 (EDT) Received: by mail-pa0-f53.google.com with SMTP id bh4so2860422pad.40 for ; Sat, 27 Apr 2013 00:40:22 -0700 (PDT) Message-ID: <517B80DD.7010008@gmail.com> Date: Sat, 27 Apr 2013 15:40:13 +0800 From: Simon Jeons MIME-Version: 1.0 Subject: Re: [question] call mark_page_accessed() in minor fault References: <20130423122542.GA5638@gmail.com> <5176866A.2060400@openvz.org> <20130423134935.GA10138@gmail.com> <517B79E6.5050204@gmail.com> <20130427075516.GA31442@gmail.com> In-Reply-To: <20130427075516.GA31442@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Zheng Liu Cc: Konstantin Khlebnikov , linux-mm@kvack.org, muming.wq@taobao.com Hi Zheng, On 04/27/2013 03:55 PM, Zheng Liu wrote: > On Sat, Apr 27, 2013 at 03:10:30PM +0800, Simon Jeons wrote: >> Hi Zheng, >> On 04/23/2013 09:49 PM, Zheng Liu wrote: >>> Hi Konstantin, >>> >>> On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote: >>>> Zheng Liu wrote: >>>>> Hi all, >>>>> >>>>> Recently we meet a performance regression about mmaped page. When we upgrade >>>>> our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel, >>>>> we will find that mmaped pages are reclaimed very quickly. We found that when >>>>> we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in >>>>> 2.6.32 kernel we don't call mark_page_accesed(). This means that mmaped pages >>>>> in 2.6.18 kernel are activated and moved into active list. While in 2.6.32 >>>>> kernel mmaped pages are still kept in inactive list. >>>>> >>>>> So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't >>>>> call it in 2.6.32 kernel. Has any reason here? >>>> Behavior was changed in commit >>>> v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path" >>> Thanks for pointing it out. >>> >>>> Please see also commits >>>> v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and >>> Yes, I will give it try. If I understand correctly, this commit is >>> useful for multi-processes program that access a shared mmaped page, >>> but that could not be useful for us because our program is multi-thread. >> What's the difference behavior between multi-processes and >> multi-thread in this case? > Hi Simon, > > Sorry, I am not a MM expert. IIUC, if we have two processes, this > mmaped page will be moved into active list. But if we only have two > threads, reference_ptes == 1, and this mmaped page won't be moved into > active list. Finally this page could be evicted. Am I missing > something? Multi-threads will have same mm_struct and task_struct? > > Thanks, > - Zheng -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx194.postini.com [74.125.245.194]) by kanga.kvack.org (Postfix) with SMTP id CF4E46B0032 for ; Sat, 27 Apr 2013 06:57:10 -0400 (EDT) Received: by mail-da0-f47.google.com with SMTP id p1so2295559dad.20 for ; Sat, 27 Apr 2013 03:57:10 -0700 (PDT) Date: Sat, 27 Apr 2013 19:14:35 +0800 From: Zheng Liu Subject: Re: [question] call mark_page_accessed() in minor fault Message-ID: <20130427111435.GA6443@gmail.com> References: <20130423122542.GA5638@gmail.com> <5176866A.2060400@openvz.org> <20130423134935.GA10138@gmail.com> <517B79E6.5050204@gmail.com> <20130427075516.GA31442@gmail.com> <517B80DD.7010008@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <517B80DD.7010008@gmail.com> Sender: owner-linux-mm@kvack.org List-ID: To: Simon Jeons Cc: Konstantin Khlebnikov , linux-mm@kvack.org, muming.wq@taobao.com On Sat, Apr 27, 2013 at 03:40:13PM +0800, Simon Jeons wrote: > Hi Zheng, > On 04/27/2013 03:55 PM, Zheng Liu wrote: > >On Sat, Apr 27, 2013 at 03:10:30PM +0800, Simon Jeons wrote: > >>Hi Zheng, > >>On 04/23/2013 09:49 PM, Zheng Liu wrote: > >>>Hi Konstantin, > >>> > >>>On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote: > >>>>Zheng Liu wrote: > >>>>>Hi all, > >>>>> > >>>>>Recently we meet a performance regression about mmaped page. When we upgrade > >>>>>our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel, > >>>>>we will find that mmaped pages are reclaimed very quickly. We found that when > >>>>>we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in > >>>>>2.6.32 kernel we don't call mark_page_accesed(). This means that mmaped pages > >>>>>in 2.6.18 kernel are activated and moved into active list. While in 2.6.32 > >>>>>kernel mmaped pages are still kept in inactive list. > >>>>> > >>>>>So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't > >>>>>call it in 2.6.32 kernel. Has any reason here? > >>>>Behavior was changed in commit > >>>>v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path" > >>>Thanks for pointing it out. > >>> > >>>>Please see also commits > >>>>v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and > >>>Yes, I will give it try. If I understand correctly, this commit is > >>>useful for multi-processes program that access a shared mmaped page, > >>>but that could not be useful for us because our program is multi-thread. > >>What's the difference behavior between multi-processes and > >>multi-thread in this case? > >Hi Simon, > > > >Sorry, I am not a MM expert. IIUC, if we have two processes, this > >mmaped page will be moved into active list. But if we only have two > >threads, reference_ptes == 1, and this mmaped page won't be moved into > >active list. Finally this page could be evicted. Am I missing > >something? > > Multi-threads will have same mm_struct and task_struct? Multi-threads share one mm_struct and have different task_struct's. Multi-processes have different mm_struct's and task_struct's. Regards, - Zheng -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org