[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[shuang.qiu]

From: Shuang Qiu <shuang.qiu@oracle.com>

In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()),
it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger
the account of lruvec_stat,and will not be free in memory pressure either.
So the testcase may fail with:
...
madvise09.c:219: INFO: Memory hungry child 6178 started, try 10
madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp
madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
...
Change the PAGES to the value more than 32 can fix such issue.

Signed-off-by: Shuang Qiu <shuang.qiu@oracle.com>
---
 testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/testcases/kernel/syscalls/madvise/madvise09.c b/testcases/kernel/syscalls/madvise/madvise09.c
index 01075f6..3759053 100644
--- a/testcases/kernel/syscalls/madvise/madvise09.c
+++ b/testcases/kernel/syscalls/madvise/madvise09.c
@@ -57,7 +57,7 @@ static int sleep_between_faults;
 
 static int swap_accounting_enabled;
 
-#define PAGES 32
+#define PAGES 64
 #define TOUCHED_PAGE1 0
 #define TOUCHED_PAGE2 10
 
-- 
1.9.1

[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[Thadeu Lima de Souza Cascardo]

On Wed, Sep 04, 2019 at 02:00:23PM +0800, shuang.qiu@oracle.com wrote:
> From: Shuang Qiu <shuang.qiu@oracle.com>
> 
> In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()),
> it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger
> the account of lruvec_stat,and will not be free in memory pressure either.
> So the testcase may fail with:
> ...
> madvise09.c:219: INFO: Memory hungry child 6178 started, try 10
> madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp
> madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
> ...
> Change the PAGES to the value more than 32 can fix such issue.
> 
> Signed-off-by: Shuang Qiu <shuang.qiu@oracle.com>

On ppc64le, I was getting retries, as OOM was getting hit. The test would
ultimately time out. This fixes it for me.

Thanks.
Cascardo.

Tested-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>

> ---
>  testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/testcases/kernel/syscalls/madvise/madvise09.c b/testcases/kernel/syscalls/madvise/madvise09.c
> index 01075f6..3759053 100644
> --- a/testcases/kernel/syscalls/madvise/madvise09.c
> +++ b/testcases/kernel/syscalls/madvise/madvise09.c
> @@ -57,7 +57,7 @@ static int sleep_between_faults;
>  
>  static int swap_accounting_enabled;
>  
> -#define PAGES 32
> +#define PAGES 64
>  #define TOUCHED_PAGE1 0
>  #define TOUCHED_PAGE2 10
>  
> -- 
> 1.9.1
> 
> 
> -- 
> Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[Li Wang]

On Wed, Sep 4, 2019 at 2:01 PM <shuang.qiu@oracle.com> wrote:
>
> From: Shuang Qiu <shuang.qiu@oracle.com>
>
> In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()),
> it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger
> the account of lruvec_stat,and will not be free in memory pressure either.
> So the testcase may fail with:
> ...
> madvise09.c:219: INFO: Memory hungry child 6178 started, try 10
> madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp
> madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
> ...
> Change the PAGES to the value more than 32 can fix such issue.
>
> Signed-off-by: Shuang Qiu <shuang.qiu@oracle.com>
> ---
>  testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/testcases/kernel/syscalls/madvise/madvise09.c b/testcases/kernel/syscalls/madvise/madvise09.c
> index 01075f6..3759053 100644
> --- a/testcases/kernel/syscalls/madvise/madvise09.c
> +++ b/testcases/kernel/syscalls/madvise/madvise09.c
> @@ -57,7 +57,7 @@ static int sleep_between_faults;
>
>  static int swap_accounting_enabled;
>
> -#define PAGES 32
> +#define PAGES 64

I'm not sure why 64 pages is a proper value? Can you explain more?

-- 
Regards,
Li Wang

[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[Shuang Qiu]

On 09/05/2019 04:29 PM, Li Wang wrote:
> On Wed, Sep 4, 2019 at 2:01 PM <shuang.qiu@oracle.com> wrote:
>> From: Shuang Qiu <shuang.qiu@oracle.com>
>>
>> In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()),
>> it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger
>> the account of lruvec_stat,and will not be free in memory pressure either.
>> So the testcase may fail with:
>> ...
>> madvise09.c:219: INFO: Memory hungry child 6178 started, try 10
>> madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp
>> madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
>> ...
>> Change the PAGES to the value more than 32 can fix such issue.
>>
>> Signed-off-by: Shuang Qiu <shuang.qiu@oracle.com>
>> ---
>>   testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/testcases/kernel/syscalls/madvise/madvise09.c b/testcases/kernel/syscalls/madvise/madvise09.c
>> index 01075f6..3759053 100644
>> --- a/testcases/kernel/syscalls/madvise/madvise09.c
>> +++ b/testcases/kernel/syscalls/madvise/madvise09.c
>> @@ -57,7 +57,7 @@ static int sleep_between_faults;
>>
>>   static int swap_accounting_enabled;
>>
>> -#define PAGES 32
>> +#define PAGES 64
> I'm not sure why 64 pages is a proper value? Can you explain more?
I think any value which larger than 32 is ok.
So I tested 64 and works fine.

Thanks
Shuang
>

[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[Li Wang]

On Thu, Sep 5, 2019 at 6:02 PM Shuang Qiu <shuang.qiu@oracle.com> wrote:
>
> On 09/05/2019 04:29 PM, Li Wang wrote:
> > On Wed, Sep 4, 2019 at 2:01 PM <shuang.qiu@oracle.com> wrote:
> >> From: Shuang Qiu <shuang.qiu@oracle.com>
> >>
> >> In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()),
> >> it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger
> >> the account of lruvec_stat,and will not be free in memory pressure either.
> >> So the testcase may fail with:
> >> ...
> >> madvise09.c:219: INFO: Memory hungry child 6178 started, try 10
> >> madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp
> >> madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
> >> ...
> >> Change the PAGES to the value more than 32 can fix such issue.
> >>
> >> Signed-off-by: Shuang Qiu <shuang.qiu@oracle.com>
> >> ---
> >>   testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/testcases/kernel/syscalls/madvise/madvise09.c b/testcases/kernel/syscalls/madvise/madvise09.c
> >> index 01075f6..3759053 100644
> >> --- a/testcases/kernel/syscalls/madvise/madvise09.c
> >> +++ b/testcases/kernel/syscalls/madvise/madvise09.c
> >> @@ -57,7 +57,7 @@ static int sleep_between_faults;
> >>
> >>   static int swap_accounting_enabled;
> >>
> >> -#define PAGES 32
> >> +#define PAGES 64
> > I'm not sure why 64 pages is a proper value? Can you explain more?
> I think any value which larger than 32 is ok.
> So I tested 64 and works fine.

I just tried with '42' randomly, but it doesn't work. I guess that
might have a waterline to trigger this memory page reclaiming.

I looked at the code calling path, it seems the 'sc->nr_reclaimed' is
the key value to control that, but still not 100% sure, I will keep
debugging it for a while.

try_to_free_mem_cgroup_pages
  do_try_to_free_pages
    shrink_zone
      shrink_node
        shrink_node_memcg
          get_scan_count
            lruvec_lru_size
              lruvec_page_state

-- 
Regards,
Li Wang

[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[Cyril Hrubis]

Hi!
> I'm not sure why 64 pages is a proper value? Can you explain more?

I've just talked to Michal Hocko and the conclusion is that we should
allocate several megabytes to avoid depending kernel implementation
details, which would mean allocating thousands of pages.

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[Li Wang]

On Wed, Sep 11, 2019 at 7:19 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > I'm not sure why 64 pages is a proper value? Can you explain more?
>
> I've just talked to Michal Hocko and the conclusion is that we should
> allocate several megabytes to avoid depending kernel implementation
> details, which would mean allocating thousands of pages.
>

Sounds reasonable. Allocating thousands of pages must trigger the MADV_FREE
page reclaiming.

@shuang.qiu, Hi Shuang, would you mind to rewrite the patch, or should I do
that?

-- 
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190912/2100c785/attachment.htm>

[LTP] [PATCH] madvise09:Change PAGES size to the value more than 32

[Shuang Qiu]

On 09/12/2019 10:51 AM, Li Wang wrote:
>
>
> On Wed, Sep 11, 2019 at 7:19 PM Cyril Hrubis <chrubis@suse.cz 
> <mailto:chrubis@suse.cz>> wrote:
>
>     Hi!
>     > I'm not sure why 64 pages is a proper value? Can you explain more?
>
>     I've just talked to Michal Hocko and the conclusion is that we should
>     allocate several megabytes to avoid depending kernel implementation
>     details, which would mean allocating thousands of pages.
>
>
> Sounds reasonable. Allocating thousands of pages must trigger 
> the?MADV_FREE page reclaiming.
>
> @shuang.qiu, Hi Shuang, would you mind to rewrite the patch, or should 
> I do that?
Thanks for helping to investigate this issue.
I will update the patch.
32 pages is really small in current system which may be held on 
reclaiming small piece of memory,thousands of pages is reasonable.

Thanks
Shuang
>
> -- 
> Regards,
> Li Wang


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190912/4465138e/attachment.htm>