* [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
@ 2021-01-27 11:56 Martin Doucha
2021-01-27 12:03 ` Cyril Hrubis
0 siblings, 1 reply; 6+ messages in thread
From: Martin Doucha @ 2021-01-27 11:56 UTC (permalink / raw)
To: ltp
tst_pollute_memory() still has OOM issues on system with huge amounts of RAM.
Set safety margin to the largest value of:
- 2 * min_free_kbytes
- 128MB
- 4096 pages
- Free RAM / 128 (to account for memory allocation overhead)
Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
lib/tst_memutils.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c
index dd09db490..7cdb3dbe0 100644
--- a/lib/tst_memutils.c
+++ b/lib/tst_memutils.c
@@ -20,8 +20,11 @@ void tst_pollute_memory(size_t maxsize, int fillchar)
struct sysinfo info;
SAFE_SYSINFO(&info);
- safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024);
+ SAFE_FILE_SCANF("/proc/sys/vm/min_free_kbytes", "%zd", &safety);
+ safety = MAX(2048 * safety, 128 * 1024 * 1024);
+ safety = MAX(safety, 4096 * SAFE_SYSCONF(_SC_PAGESIZE));
safety /= info.mem_unit;
+ safety = MAX(safety, info.freeram / 128);
if (info.freeswap > safety)
safety = 0;
--
2.29.2
^ permalink raw reply related [flat|nested] 6+ messages in thread* [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
2021-01-27 11:56 [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems Martin Doucha
@ 2021-01-27 12:03 ` Cyril Hrubis
2021-01-27 12:14 ` liuxp11
0 siblings, 1 reply; 6+ messages in thread
From: Cyril Hrubis @ 2021-01-27 12:03 UTC (permalink / raw)
To: ltp
Hi!
Actually I've been thinking about this one and since this is a strictly
best effort action I think we should be more aggressive about the safety
margin.
I also started to talk to kernel developers about this and generally
about tests that try to allocate maximal amount of memory without
triggering OOM since we have a few in the tree. I will send the results
of this discussion once I have any here as well.
--
Cyril Hrubis
chrubis@suse.cz
^ permalink raw reply [flat|nested] 6+ messages in thread
* [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
2021-01-27 12:03 ` Cyril Hrubis
@ 2021-01-27 12:14 ` liuxp11
2021-01-27 12:28 ` Cyril Hrubis
0 siblings, 1 reply; 6+ messages in thread
From: liuxp11 @ 2021-01-27 12:14 UTC (permalink / raw)
To: ltp
According my test,malloc amount greater than available memory,then invoke the oom-killer.
#man free
Estimation of how much memory is available for starting new applications, without swapping.
[root@liuxp mywork]# head /proc/meminfo
MemTotal: 198101744 kB
MemFree: 189303148 kB
MemAvailable: 188566732 kB
Set safety margin to larger value, may be not sensible.
From: Cyril Hrubis
Date: 2021-01-27 20:03
To: Martin Doucha
CC: liuxp11; ltp
Subject: Re: [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
Hi!
Actually I've been thinking about this one and since this is a strictly
best effort action I think we should be more aggressive about the safety
margin.
I also started to talk to kernel developers about this and generally
about tests that try to allocate maximal amount of memory without
triggering OOM since we have a few in the tree. I will send the results
of this discussion once I have any here as well.
--
Cyril Hrubis
chrubis@suse.cz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/ddde8d39/attachment.htm>
^ permalink raw reply [flat|nested] 6+ messages in thread* [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
2021-01-27 12:14 ` liuxp11
@ 2021-01-27 12:28 ` Cyril Hrubis
2021-01-27 12:30 ` liuxp11
2021-01-27 13:05 ` Li Wang
0 siblings, 2 replies; 6+ messages in thread
From: Cyril Hrubis @ 2021-01-27 12:28 UTC (permalink / raw)
To: ltp
Hi!
> According my test,malloc amount greater than available memory,then invoke the oom-killer.
However that does not mean that allocating MemAvailable or slightly less
than MemAvailable is safe. In fact it's not, at least that I've been
told be kernel developers. I will try to figure out something safe
enough with their help.
--
Cyril Hrubis
chrubis@suse.cz
^ permalink raw reply [flat|nested] 6+ messages in thread
* [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
2021-01-27 12:28 ` Cyril Hrubis
@ 2021-01-27 12:30 ` liuxp11
2021-01-27 13:05 ` Li Wang
1 sibling, 0 replies; 6+ messages in thread
From: liuxp11 @ 2021-01-27 12:30 UTC (permalink / raw)
To: ltp
OK,I also wanna know how to fix it.
Thanks!
From: Cyril Hrubis
Date: 2021-01-27 20:28
To: liuxp11@chinatelecom.cn
CC: mdoucha; ltp
Subject: Re: Re: [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
Hi!
> According my test,malloc amount greater than available memory,then invoke the oom-killer.
However that does not mean that allocating MemAvailable or slightly less
than MemAvailable is safe. In fact it's not, at least that I've been
told be kernel developers. I will try to figure out something safe
enough with their help.
--
Cyril Hrubis
chrubis@suse.cz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/ed42b7e7/attachment.htm>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
2021-01-27 12:28 ` Cyril Hrubis
2021-01-27 12:30 ` liuxp11
@ 2021-01-27 13:05 ` Li Wang
1 sibling, 0 replies; 6+ messages in thread
From: Li Wang @ 2021-01-27 13:05 UTC (permalink / raw)
To: ltp
On Wed, Jan 27, 2021 at 8:27 PM Cyril Hrubis <chrubis@suse.cz> wrote:
> Hi!
> > According my test,malloc amount greater than available memory,then
> invoke the oom-killer.
>
> However that does not mean that allocating MemAvailable or slightly less
> than MemAvailable is safe. In fact it's not, at least that I've been
> told be kernel developers. I will try to figure out something safe
> enough with their help.
>
If we only hope to avoid trigger OOM during allocating until the
MAP_FAILED, I think the knob 'proc/sys/vm/overcommit_memory'
maybe helpful.
e.g.
set overcommit_memory = 2, overcommit_ratio = 85 or 90;
... memory allocating as much as you can ...
restore the value of overcommit_*
--
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/dbf6179c/attachment-0001.htm>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-01-27 13:05 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-01-27 11:56 [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems Martin Doucha
2021-01-27 12:03 ` Cyril Hrubis
2021-01-27 12:14 ` liuxp11
2021-01-27 12:28 ` Cyril Hrubis
2021-01-27 12:30 ` liuxp11
2021-01-27 13:05 ` Li Wang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.