[LTP] [PATCH RFC 1/2] lib: add SAFE_READ_PROC_STATUS macro

[LTP] [PATCH RFC 1/2] lib: add SAFE_READ_PROC_STATUS macro

[Li Wang]

To parse /proc/PID/status files, for example:
    SAFE_READ_PROC_STATUS(pid, "VmSwap:");

Signed-off-by: Li Wang <liwang@redhat.com>
Cc: Petr Vorel <pvorel@suse.cz>
Cc: Alexander Egorenkov <egorenar@linux.ibm.com>
---
 include/tst_safe_file_ops.h | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/include/tst_safe_file_ops.h b/include/tst_safe_file_ops.h
index 894c16123..7a4076c4c 100644
--- a/include/tst_safe_file_ops.h
+++ b/include/tst_safe_file_ops.h
@@ -25,6 +25,14 @@
                         &tst_rval); \
         tst_rval;})
 
+#define SAFE_READ_PROC_STATUS(pid, item) \
+       ({long tst_rval_; \
+        char tst_path_[128]; \
+        sprintf(tst_path_, "/proc/%d/status", pid); \
+        SAFE_FILE_LINES_SCANF(tst_path_, item " %ld", \
+                        &tst_rval_); \
+        tst_rval_;})
+
 #define FILE_PRINTF(path, fmt, ...) \
 	file_printf(__FILE__, __LINE__, \
 		    (path), (fmt), ## __VA_ARGS__)
-- 
2.21.3

[LTP] [PATCH RFC 2/2] swapping01: check memory swap usage per process

[Li Wang]

Since previously swapping01 read the system FreeSwap for counting
usage of swap-size, that's not precise on system especially with
eating-memory daemon?in the background. Now, we try to check the
'VmmSwap' in proc/PID/status?per process, to get rid of?the potential
influence from?other processes?which easily leads to false positive.

Signed-off-by: Li Wang <liwang@redhat.com>
Cc: Petr Vorel <pvorel@suse.cz>
Cc: Alexander Egorenkov <egorenar@linux.ibm.com>
---
 testcases/kernel/mem/swapping/swapping01.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/testcases/kernel/mem/swapping/swapping01.c b/testcases/kernel/mem/swapping/swapping01.c
index 8106f6466..0f693f313 100644
--- a/testcases/kernel/mem/swapping/swapping01.c
+++ b/testcases/kernel/mem/swapping/swapping01.c
@@ -137,8 +137,7 @@ static void check_swapping(void)
 		i++;
 	}
 
-	swap_free_now = SAFE_READ_MEMINFO("SwapFree:");
-	swapped = swap_free_init - swap_free_now;
+	swapped = SAFE_READ_PROC_STATUS(pid, "VmSwap:");
 	if (swapped > mem_over_max) {
 		kill(pid, SIGCONT);
 		tst_brk(TFAIL, "heavy swapping detected: "
-- 
2.21.3

[LTP] [PATCH RFC 2/2] swapping01: check memory swap usage per process

[Petr Vorel]

Hi Li,

> Since previously swapping01 read the system FreeSwap for counting
> usage of swap-size, that's not precise on system especially with
> eating-memory daemon?in the background. Now, we try to check the
> 'VmmSwap' in proc/PID/status?per process, to get rid of?the potential
> influence from?other processes?which easily leads to false positive.

LGTM.

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

> Signed-off-by: Li Wang <liwang@redhat.com>
> Cc: Petr Vorel <pvorel@suse.cz>
> Cc: Alexander Egorenkov <egorenar@linux.ibm.com>
> ---
>  testcases/kernel/mem/swapping/swapping01.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)

> diff --git a/testcases/kernel/mem/swapping/swapping01.c b/testcases/kernel/mem/swapping/swapping01.c
> index 8106f6466..0f693f313 100644
> --- a/testcases/kernel/mem/swapping/swapping01.c
> +++ b/testcases/kernel/mem/swapping/swapping01.c
> @@ -137,8 +137,7 @@ static void check_swapping(void)
>  		i++;
>  	}

> -	swap_free_now = SAFE_READ_MEMINFO("SwapFree:");
> -	swapped = swap_free_init - swap_free_now;
> +	swapped = SAFE_READ_PROC_STATUS(pid, "VmSwap:");
>  	if (swapped > mem_over_max) {
>  		kill(pid, SIGCONT);
>  		tst_brk(TFAIL, "heavy swapping detected: "

[LTP] [PATCH RFC 2/2] swapping01: check memory swap usage per process

[Cyril Hrubis]

Hi!
> Since previously swapping01 read the system FreeSwap for counting
> usage of swap-size, that's not precise on system especially with
> eating-memory daemon??in the background. Now, we try to check the
> 'VmmSwap' in proc/PID/status??per process, to get rid of??the potential
> influence from??other processes??which easily leads to false positive.

I've been looking for a while at the kernel commit:

commit 50a15981a1fac7e019ff7c3cba87531fb580f065
Author: Martin Schwidefsky <schwidefsky@de.ibm.com>
Date:   Sun Jul 24 10:47:59 2011 +0200

    [S390] reference bit testing for unmapped pages

For which this test seems to be a reproducer and as far as I can tell
this fix is not correct.

If I understand correctly what we are trying to test here is that the
kernel will not attempt to swap out unreferenced pages, so we have to,
by definition, look at the system counters not the process ones.

-- 
Cyril Hrubis
chrubis@suse.cz

[LTP] [PATCH RFC 2/2] swapping01: check memory swap usage per process

[Li Wang]

Hi Cyril,

On Mon, Jan 25, 2021 at 11:24 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > Since previously swapping01 read the system FreeSwap for counting
> > usage of swap-size, that's not precise on system especially with
> > eating-memory daemon??in the background. Now, we try to check the
> > 'VmmSwap' in proc/PID/status??per process, to get rid of??the potential
> > influence from??other processes??which easily leads to false positive.
>
> I've been looking for a while at the kernel commit:
>
> commit 50a15981a1fac7e019ff7c3cba87531fb580f065
> Author: Martin Schwidefsky <schwidefsky@de.ibm.com>
> Date:   Sun Jul 24 10:47:59 2011 +0200
>
>     [S390] reference bit testing for unmapped pages
>
> For which this test seems to be a reproducer and as far as I can tell
> this fix is not correct.
>
> If I understand correctly what we are trying to test here is that the
> kernel will not attempt to swap out unreferenced pages, so we have to,
> by definition, look at the system counters not the process ones.
>

Thanks for point out this! Seems we were all working towards
making the test robust but neglect it's a reproducer:).


As we have discussed many unsure things will take side effect to
the system swap-counting, that's what we do NOT want to expect.
Maybe, can we encapsulate all of the means in a process to avoid
involving the whole system swap-counting. e.g.

Child:
         to touch unreferenced pages (via
alloc&write&free 1.3*MemAvailable) [1]
         alloc&wirte 1.3*MemAvailable
         raise(SIGSTOP)
         ...
Parent:
         waiting for child suspension
         check child's VmSwap to see if heavy-swap occurs in a process
         ...

Does this make some sense to the test or, any else suggestion?

[1] As to perform alloc&write&free, the system pages will go through a
completed life cycle from buddy-system to active-list to inactive-list
then back to buddy-system, which reflect to a page status is theoretically
like:
"inactive,unreferenced -> active,referenced -> ... ->inactive,unreferenced"
so that might helpful to produce what the kernel target commit fixed
situation.

-- 
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210126/8da66ebe/attachment.htm>

[LTP] [PATCH RFC 2/2] swapping01: check memory swap usage per process

[Li Wang]

Hi,

On Tue, Jan 26, 2021 at 1:25 PM Li Wang <liwang@redhat.com> wrote:

> Hi Cyril,
>
> On Mon, Jan 25, 2021 at 11:24 PM Cyril Hrubis <chrubis@suse.cz> wrote:
>
>> Hi!
>> > Since previously swapping01 read the system FreeSwap for counting
>> > usage of swap-size, that's not precise on system especially with
>> > eating-memory daemon??in the background. Now, we try to check the
>> > 'VmmSwap' in proc/PID/status??per process, to get rid of??the potential
>> > influence from??other processes??which easily leads to false positive.
>>
>> I've been looking for a while at the kernel commit:
>>
>> commit 50a15981a1fac7e019ff7c3cba87531fb580f065
>> Author: Martin Schwidefsky <schwidefsky@de.ibm.com>
>> Date:   Sun Jul 24 10:47:59 2011 +0200
>>
>>     [S390] reference bit testing for unmapped pages
>>
>> For which this test seems to be a reproducer and as far as I can tell
>> this fix is not correct.
>>
>> If I understand correctly what we are trying to test here is that the
>> kernel will not attempt to swap out unreferenced pages, so we have to,
>> by definition, look at the system counters not the process ones.
>>
>
> Thanks for point out this! Seems we were all working towards
> making the test robust but neglect it's a reproducer:).
>
>
> As we have discussed many unsure things will take side effect to
> the system swap-counting, that's what we do NOT want to expect.
> Maybe, can we encapsulate all of the means in a process to avoid
> involving the whole system swap-counting. e.g.
>
> Child:
>          to touch unreferenced pages (via
> alloc&write&free 1.3*MemAvailable) [1]
>          alloc&wirte 1.3*MemAvailable
>          raise(SIGSTOP)
>          ...
> Parent:
>          waiting for child suspension
>          check child's VmSwap to see if heavy-swap occurs in a process
>          ...
>
> Does this make some sense to the test or, any else suggestion?
>
> [1] As to perform alloc&write&free, the system pages will go through a
> completed life cycle from buddy-system to active-list to inactive-list
> then back to buddy-system, which reflect to a page status is theoretically
> like:
> "inactive,unreferenced -> active,referenced -> ... ->inactive,unreferenced"
> so that might helpful to produce what the kernel target commit fixed
> situation.
>

To verify this assumption, I run the swapping01 with/without my patch-V2
against an unfix-kernel(2.6.39). Seems swap out unreferenced pages also
could be occurred in a single process, which gives me some confidence to
send out my patch V2.

-------
Reproduce the bug easily with an unfix-kernel

# uname  -r
2.6.39

# free -m
             total       used       free     shared    buffers     cached
Mem:          2006       1559        447          0         22       1359
-/+ buffers/cache:        178       1828
Swap:         4031         12       4019

# ./swapping01
tst_test.c:1263: TINFO: Timeout per run is 0h 05m 00s
swapping01.c:110: TINFO: available physical memory: 1896 MB
swapping01.c:113: TINFO: try to allocate: 2466 MB
swapping01.c:148: TFAIL: heavy swapping detected: 1905 MB swapped.

-- 
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210224/24561ebb/attachment-0001.htm>

[LTP] [PATCH RFC 1/2] lib: add SAFE_READ_PROC_STATUS macro

[Petr Vorel]

> To parse /proc/PID/status files, for example:
>     SAFE_READ_PROC_STATUS(pid, "VmSwap:");

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr