[question] call mark_page_accessed() in minor fault

[question] call mark_page_accessed() in minor fault

[Zheng Liu]

Hi all,

Recently we meet a performance regression about mmaped page.  When we upgrade
our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel,
we will find that mmaped pages are reclaimed very quickly.  We found that when
we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in
2.6.32 kernel we don't call mark_page_accesed().  This means that mmaped pages
in 2.6.18 kernel are activated and moved into active list.  While in 2.6.32
kernel mmaped pages are still kept in inactive list.

So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't
call it in 2.6.32 kernel.  Has any reason here?

Thanks in advance,
						- Zheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [question] call mark_page_accessed() in minor fault

[Konstantin Khlebnikov]

Zheng Liu wrote:
> Hi all,
>
> Recently we meet a performance regression about mmaped page.  When we upgrade
> our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel,
> we will find that mmaped pages are reclaimed very quickly.  We found that when
> we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in
> 2.6.32 kernel we don't call mark_page_accesed().  This means that mmaped pages
> in 2.6.18 kernel are activated and moved into active list.  While in 2.6.32
> kernel mmaped pages are still kept in inactive list.
>
> So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't
> call it in 2.6.32 kernel.  Has any reason here?

Behavior was changed in commit
v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path"

Please see also commits
v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and
v3.2-4877-gc909e99 "vmscan: activate executable pages after first usage".
Probably they can solve some of your problems.

>
> Thanks in advance,
> 						- Zheng
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email:<a href=mailto:"dont@kvack.org">  email@kvack.org</a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [question] call mark_page_accessed() in minor fault

[Zheng Liu]

Hi Konstantin,

On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote:
> Zheng Liu wrote:
> >Hi all,
> >
> >Recently we meet a performance regression about mmaped page.  When we upgrade
> >our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel,
> >we will find that mmaped pages are reclaimed very quickly.  We found that when
> >we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in
> >2.6.32 kernel we don't call mark_page_accesed().  This means that mmaped pages
> >in 2.6.18 kernel are activated and moved into active list.  While in 2.6.32
> >kernel mmaped pages are still kept in inactive list.
> >
> >So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't
> >call it in 2.6.32 kernel.  Has any reason here?
> 
> Behavior was changed in commit
> v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path"

Thanks for pointing it out.

> 
> Please see also commits
> v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and

Yes, I will give it try.  If I understand correctly, this commit is
useful for multi-processes program that access a shared mmaped page,
but that could not be useful for us because our program is multi-thread.

> v3.2-4877-gc909e99 "vmscan: activate executable pages after first usage".

We have backported this patch, but it is useless.  This commit only
tries to activate a executable page, but our mmaped pages aren't with
this flag.

Additional question is that currently mmaped page is reclaimed too
quickly.  I think maybe we need to adjust our page reclaim strategy to
balance number of pages between mmaped page and file page cache.  Now
every time we access a page using read(2)/write(2), this page will be
touched.  But after first time we touch a mmaped page, we never touch it
again except that this page is evicted.

Regards,
                                                - Zheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [question] call mark_page_accessed() in minor fault

[Simon Jeons]

Hi Zheng,
On 04/23/2013 09:49 PM, Zheng Liu wrote:
> Hi Konstantin,
>
> On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote:
>> Zheng Liu wrote:
>>> Hi all,
>>>
>>> Recently we meet a performance regression about mmaped page.  When we upgrade
>>> our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel,
>>> we will find that mmaped pages are reclaimed very quickly.  We found that when
>>> we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in
>>> 2.6.32 kernel we don't call mark_page_accesed().  This means that mmaped pages
>>> in 2.6.18 kernel are activated and moved into active list.  While in 2.6.32
>>> kernel mmaped pages are still kept in inactive list.
>>>
>>> So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't
>>> call it in 2.6.32 kernel.  Has any reason here?
>> Behavior was changed in commit
>> v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path"
> Thanks for pointing it out.
>
>> Please see also commits
>> v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and
> Yes, I will give it try.  If I understand correctly, this commit is
> useful for multi-processes program that access a shared mmaped page,
> but that could not be useful for us because our program is multi-thread.

What's the difference behavior between multi-processes and multi-thread 
in this case?

>
>> v3.2-4877-gc909e99 "vmscan: activate executable pages after first usage".
> We have backported this patch, but it is useless.  This commit only
> tries to activate a executable page, but our mmaped pages aren't with
> this flag.
>
> Additional question is that currently mmaped page is reclaimed too
> quickly.  I think maybe we need to adjust our page reclaim strategy to
> balance number of pages between mmaped page and file page cache.  Now
> every time we access a page using read(2)/write(2), this page will be
> touched.  But after first time we touch a mmaped page, we never touch it
> again except that this page is evicted.
>
> Regards,
>                                                  - Zheng
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [question] call mark_page_accessed() in minor fault

[Simon Jeons]

Hi Zheng,
On 04/27/2013 03:55 PM, Zheng Liu wrote:
> On Sat, Apr 27, 2013 at 03:10:30PM +0800, Simon Jeons wrote:
>> Hi Zheng,
>> On 04/23/2013 09:49 PM, Zheng Liu wrote:
>>> Hi Konstantin,
>>>
>>> On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote:
>>>> Zheng Liu wrote:
>>>>> Hi all,
>>>>>
>>>>> Recently we meet a performance regression about mmaped page.  When we upgrade
>>>>> our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel,
>>>>> we will find that mmaped pages are reclaimed very quickly.  We found that when
>>>>> we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in
>>>>> 2.6.32 kernel we don't call mark_page_accesed().  This means that mmaped pages
>>>>> in 2.6.18 kernel are activated and moved into active list.  While in 2.6.32
>>>>> kernel mmaped pages are still kept in inactive list.
>>>>>
>>>>> So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't
>>>>> call it in 2.6.32 kernel.  Has any reason here?
>>>> Behavior was changed in commit
>>>> v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path"
>>> Thanks for pointing it out.
>>>
>>>> Please see also commits
>>>> v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and
>>> Yes, I will give it try.  If I understand correctly, this commit is
>>> useful for multi-processes program that access a shared mmaped page,
>>> but that could not be useful for us because our program is multi-thread.
>> What's the difference behavior between multi-processes and
>> multi-thread in this case?
> Hi Simon,
>
> Sorry, I am not a MM expert.  IIUC, if we have two processes, this
> mmaped page will be moved into active list.  But if we only have two
> threads, reference_ptes == 1, and this mmaped page won't be moved into
> active list.  Finally this page could be evicted.  Am I missing
> something?

Multi-threads will have same mm_struct and task_struct?

>
> Thanks,
>                                                  - Zheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [question] call mark_page_accessed() in minor fault

[Zheng Liu]

On Sat, Apr 27, 2013 at 03:10:30PM +0800, Simon Jeons wrote:
> Hi Zheng,
> On 04/23/2013 09:49 PM, Zheng Liu wrote:
> >Hi Konstantin,
> >
> >On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote:
> >>Zheng Liu wrote:
> >>>Hi all,
> >>>
> >>>Recently we meet a performance regression about mmaped page.  When we upgrade
> >>>our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel,
> >>>we will find that mmaped pages are reclaimed very quickly.  We found that when
> >>>we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in
> >>>2.6.32 kernel we don't call mark_page_accesed().  This means that mmaped pages
> >>>in 2.6.18 kernel are activated and moved into active list.  While in 2.6.32
> >>>kernel mmaped pages are still kept in inactive list.
> >>>
> >>>So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't
> >>>call it in 2.6.32 kernel.  Has any reason here?
> >>Behavior was changed in commit
> >>v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path"
> >Thanks for pointing it out.
> >
> >>Please see also commits
> >>v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and
> >Yes, I will give it try.  If I understand correctly, this commit is
> >useful for multi-processes program that access a shared mmaped page,
> >but that could not be useful for us because our program is multi-thread.
> 
> What's the difference behavior between multi-processes and
> multi-thread in this case?

Hi Simon,

Sorry, I am not a MM expert.  IIUC, if we have two processes, this
mmaped page will be moved into active list.  But if we only have two
threads, reference_ptes == 1, and this mmaped page won't be moved into
active list.  Finally this page could be evicted.  Am I missing
something?

Thanks,
                                                - Zheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [question] call mark_page_accessed() in minor fault

[Zheng Liu]

On Sat, Apr 27, 2013 at 03:40:13PM +0800, Simon Jeons wrote:
> Hi Zheng,
> On 04/27/2013 03:55 PM, Zheng Liu wrote:
> >On Sat, Apr 27, 2013 at 03:10:30PM +0800, Simon Jeons wrote:
> >>Hi Zheng,
> >>On 04/23/2013 09:49 PM, Zheng Liu wrote:
> >>>Hi Konstantin,
> >>>
> >>>On Tue, Apr 23, 2013 at 05:02:34PM +0400, Konstantin Khlebnikov wrote:
> >>>>Zheng Liu wrote:
> >>>>>Hi all,
> >>>>>
> >>>>>Recently we meet a performance regression about mmaped page.  When we upgrade
> >>>>>our product system from 2.6.18 kernel to a latest kernel, such as 2.6.32 kernel,
> >>>>>we will find that mmaped pages are reclaimed very quickly.  We found that when
> >>>>>we hit a minor fault mark_page_accessed() is called in 2.6.18 kernel, but in
> >>>>>2.6.32 kernel we don't call mark_page_accesed().  This means that mmaped pages
> >>>>>in 2.6.18 kernel are activated and moved into active list.  While in 2.6.32
> >>>>>kernel mmaped pages are still kept in inactive list.
> >>>>>
> >>>>>So my question is why we call mark_page_accessed() in 2.6.18 kernel, but don't
> >>>>>call it in 2.6.32 kernel.  Has any reason here?
> >>>>Behavior was changed in commit
> >>>>v2.6.28-6130-gbf3f3bc "mm: don't mark_page_accessed in fault path"
> >>>Thanks for pointing it out.
> >>>
> >>>>Please see also commits
> >>>>v3.2-4876-g34dbc67 "vmscan: promote shared file mapped pages" and
> >>>Yes, I will give it try.  If I understand correctly, this commit is
> >>>useful for multi-processes program that access a shared mmaped page,
> >>>but that could not be useful for us because our program is multi-thread.
> >>What's the difference behavior between multi-processes and
> >>multi-thread in this case?
> >Hi Simon,
> >
> >Sorry, I am not a MM expert.  IIUC, if we have two processes, this
> >mmaped page will be moved into active list.  But if we only have two
> >threads, reference_ptes == 1, and this mmaped page won't be moved into
> >active list.  Finally this page could be evicted.  Am I missing
> >something?
> 
> Multi-threads will have same mm_struct and task_struct?

Multi-threads share one mm_struct and have different task_struct's.
Multi-processes have different mm_struct's and task_struct's.

Regards,
                                                - Zheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>