* [LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test
@ 2026-01-29 10:00 Li Wang via ltp
2026-01-29 10:08 ` Li Wang via ltp
0 siblings, 1 reply; 6+ messages in thread
From: Li Wang via ltp @ 2026-01-29 10:00 UTC (permalink / raw)
To: ltp
As each iteration of mmap_thread() grabs a fresh 16 MiB MAP_POPULATE
region and never releases it. As the loop runs, those regions
accumulate consuming both virtual address space and committed physical
memory right away instead of lazily.
Easily mmap() fails with ENOMEM on smaller/limit RAM resource system.
Error Log:
cve-2017-17052.c:48: TBROK: mmap((nil),16777216,PROT_READ(1),32802,-1,0) failed: ENOMEM (12)
tst_test.c:479: TINFO: Child process reported TBROK killing the test
tst_test.c:1909: TINFO: Killed the leftover descendant processes
Consider there is no practical upper bound on this allocation pattern,
so setting .test.min_mem_avail may not helps. Here we just tolerate
ENOMEM during the mmap_thread() looping.
Signed-off-by: Li Wang <liwang@redhat.com>
---
testcases/cve/cve-2017-17052.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/testcases/cve/cve-2017-17052.c b/testcases/cve/cve-2017-17052.c
index 700eb782e..61032c197 100644
--- a/testcases/cve/cve-2017-17052.c
+++ b/testcases/cve/cve-2017-17052.c
@@ -44,9 +44,16 @@ static void cleanup(void)
static void *mmap_thread(void *arg)
{
+ void *ptr;
+
for (;;) {
- SAFE_MMAP(NULL, 0x1000000, PROT_READ,
+ ptr = mmap(NULL, 0x1000000, PROT_READ,
MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
+
+ if ((ptr == MAP_FAILED) && (errno == ENOMEM)) {
+ usleep(1000);
+ continue;
+ }
}
return arg;
--
2.52.0
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test
2026-01-29 10:00 [LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test Li Wang via ltp
@ 2026-01-29 10:08 ` Li Wang via ltp
2026-01-29 13:28 ` Petr Vorel
2026-01-29 14:24 ` Cyril Hrubis
0 siblings, 2 replies; 6+ messages in thread
From: Li Wang via ltp @ 2026-01-29 10:08 UTC (permalink / raw)
To: Li Wang; +Cc: ltp
On Thu, Jan 29, 2026 at 6:00 PM Li Wang via ltp <ltp@lists.linux.it> wrote:
>
> As each iteration of mmap_thread() grabs a fresh 16 MiB MAP_POPULATE
> region and never releases it. As the loop runs, those regions
> accumulate consuming both virtual address space and committed physical
> memory right away instead of lazily.
>
> Easily mmap() fails with ENOMEM on smaller/limit RAM resource system.
>
> Error Log:
>
> cve-2017-17052.c:48: TBROK: mmap((nil),16777216,PROT_READ(1),32802,-1,0) failed: ENOMEM (12)
> tst_test.c:479: TINFO: Child process reported TBROK killing the test
> tst_test.c:1909: TINFO: Killed the leftover descendant processes
>
> Consider there is no practical upper bound on this allocation pattern,
> so setting .test.min_mem_avail may not helps. Here we just tolerate
> ENOMEM during the mmap_thread() looping.
>
> Signed-off-by: Li Wang <liwang@redhat.com>
> ---
> testcases/cve/cve-2017-17052.c | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/testcases/cve/cve-2017-17052.c b/testcases/cve/cve-2017-17052.c
> index 700eb782e..61032c197 100644
> --- a/testcases/cve/cve-2017-17052.c
> +++ b/testcases/cve/cve-2017-17052.c
> @@ -44,9 +44,16 @@ static void cleanup(void)
>
> static void *mmap_thread(void *arg)
> {
> + void *ptr;
> +
> for (;;) {
> - SAFE_MMAP(NULL, 0x1000000, PROT_READ,
> + ptr = mmap(NULL, 0x1000000, PROT_READ,
> MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
> +
> + if ((ptr == MAP_FAILED) && (errno == ENOMEM)) {
> + usleep(1000);
> + continue;
> + }
Oops, I forgot to handle the other failures, so the patch should be:
--- a/testcases/cve/cve-2017-17052.c
+++ b/testcases/cve/cve-2017-17052.c
@@ -44,9 +44,19 @@ static void cleanup(void)
static void *mmap_thread(void *arg)
{
+ void *ptr;
+
for (;;) {
- SAFE_MMAP(NULL, 0x1000000, PROT_READ,
+ ptr = mmap(NULL, 0x1000000, PROT_READ,
MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
+
+ if (ptr == MAP_FAILED) {
+ if (errno == ENOMEM) {
+ usleep(1000);
+ continue;
+ }
+ tst_brk(TBROK | TTERRNO, "Unexpected mmap() error");
+ }
}
return arg;
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test
2026-01-29 10:08 ` Li Wang via ltp
@ 2026-01-29 13:28 ` Petr Vorel
2026-01-29 13:44 ` Li Wang via ltp
2026-01-29 14:24 ` Cyril Hrubis
1 sibling, 1 reply; 6+ messages in thread
From: Petr Vorel @ 2026-01-29 13:28 UTC (permalink / raw)
To: Li Wang; +Cc: Martin Doucha, ltp
Hi Li,
[ Cc others as it's a pre-release ]
> On Thu, Jan 29, 2026 at 6:00 PM Li Wang via ltp <ltp@lists.linux.it> wrote:
> > As each iteration of mmap_thread() grabs a fresh 16 MiB MAP_POPULATE
> > region and never releases it. As the loop runs, those regions
> > accumulate consuming both virtual address space and committed physical
> > memory right away instead of lazily.
> > Easily mmap() fails with ENOMEM on smaller/limit RAM resource system.
> > Error Log:
> > cve-2017-17052.c:48: TBROK: mmap((nil),16777216,PROT_READ(1),32802,-1,0) failed: ENOMEM (12)
> > tst_test.c:479: TINFO: Child process reported TBROK killing the test
> > tst_test.c:1909: TINFO: Killed the leftover descendant processes
> > Consider there is no practical upper bound on this allocation pattern,
> > so setting .test.min_mem_avail may not helps. Here we just tolerate
> > ENOMEM during the mmap_thread() looping.
> > Signed-off-by: Li Wang <liwang@redhat.com>
> > ---
> > testcases/cve/cve-2017-17052.c | 9 ++++++++-
> > 1 file changed, 8 insertions(+), 1 deletion(-)
> > diff --git a/testcases/cve/cve-2017-17052.c b/testcases/cve/cve-2017-17052.c
> > index 700eb782e..61032c197 100644
> > --- a/testcases/cve/cve-2017-17052.c
> > +++ b/testcases/cve/cve-2017-17052.c
> > @@ -44,9 +44,16 @@ static void cleanup(void)
> > static void *mmap_thread(void *arg)
> > {
> > + void *ptr;
> > +
> > for (;;) {
> > - SAFE_MMAP(NULL, 0x1000000, PROT_READ,
> > + ptr = mmap(NULL, 0x1000000, PROT_READ,
> > MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
> > +
> > + if ((ptr == MAP_FAILED) && (errno == ENOMEM)) {
> > + usleep(1000);
> > + continue;
> > + }
> Oops, I forgot to handle the other failures, so the patch should be:
> --- a/testcases/cve/cve-2017-17052.c
> +++ b/testcases/cve/cve-2017-17052.c
> @@ -44,9 +44,19 @@ static void cleanup(void)
> static void *mmap_thread(void *arg)
> {
> + void *ptr;
> +
> for (;;) {
> - SAFE_MMAP(NULL, 0x1000000, PROT_READ,
> + ptr = mmap(NULL, 0x1000000, PROT_READ,
> MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
> +
> + if (ptr == MAP_FAILED) {
> + if (errno == ENOMEM) {
> + usleep(1000);
> + continue;
> + }
> + tst_brk(TBROK | TTERRNO, "Unexpected mmap() error");
> + }
> }
> return arg;
Patched version LGTM.
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Out of curiosity how much RAM is not enough for this? I tried 400 MB + eating
other RAM but was not able to trigger.
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test
2026-01-29 13:28 ` Petr Vorel
@ 2026-01-29 13:44 ` Li Wang via ltp
0 siblings, 0 replies; 6+ messages in thread
From: Li Wang via ltp @ 2026-01-29 13:44 UTC (permalink / raw)
To: Petr Vorel; +Cc: Martin Doucha, ltp
> Out of curiosity how much RAM is not enough for this? I tried 400 MB + eating
> other RAM but was not able to trigger.
It occurs (but rarely) in our CI jobs, and is hard to reproduce manually.
But if you limit resources in a cgroup or prolong the looping time for a while,
that would be possible reproduce the fail.
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test
2026-01-29 10:08 ` Li Wang via ltp
2026-01-29 13:28 ` Petr Vorel
@ 2026-01-29 14:24 ` Cyril Hrubis
2026-01-30 0:01 ` Li Wang via ltp
1 sibling, 1 reply; 6+ messages in thread
From: Cyril Hrubis @ 2026-01-29 14:24 UTC (permalink / raw)
To: Li Wang; +Cc: ltp
Hi!
> for (;;) {
> - SAFE_MMAP(NULL, 0x1000000, PROT_READ,
> + ptr = mmap(NULL, 0x1000000, PROT_READ,
> MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
> +
> + if (ptr == MAP_FAILED) {
> + if (errno == ENOMEM) {
> + usleep(1000);
> + continue;
> + }
> + tst_brk(TBROK | TTERRNO, "Unexpected mmap() error");
Maybe this part would be more readable with an else branch:
if (errno == ENOMEM)
usleep(1000);
else
tst_brk(TBROK | TTERRNO, "mmap() failed");
Otherwise:
Reviewed-by: Cyril Hrubis <chrubis@suse.cz>
> + }
> }
>
> return arg;
>
>
>
> --
> Regards,
> Li Wang
>
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp
--
Cyril Hrubis
chrubis@suse.cz
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test
2026-01-29 14:24 ` Cyril Hrubis
@ 2026-01-30 0:01 ` Li Wang via ltp
0 siblings, 0 replies; 6+ messages in thread
From: Li Wang via ltp @ 2026-01-30 0:01 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
On Thu, Jan 29, 2026 at 10:23 PM Cyril Hrubis <chrubis@suse.cz> wrote:
> Hi!
> > for (;;) {
> > - SAFE_MMAP(NULL, 0x1000000, PROT_READ,
> > + ptr = mmap(NULL, 0x1000000, PROT_READ,
> > MAP_POPULATE|MAP_ANONYMOUS|MAP_PRIVATE,
> -1, 0);
> > +
> > + if (ptr == MAP_FAILED) {
> > + if (errno == ENOMEM) {
> > + usleep(1000);
> > + continue;
> > + }
> > + tst_brk(TBROK | TTERRNO, "Unexpected mmap()
> error");
>
> Maybe this part would be more readable with an else branch:
>
> if (errno == ENOMEM)
> usleep(1000);
> else
> tst_brk(TBROK | TTERRNO, "mmap() failed");
>
Good suggestion, patch merged like this. Thanks!
--
Regards,
Li Wang
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-01-30 0:01 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-01-29 10:00 [LTP] [PATCH] cve-2017-17052: tolerate ENOMEM during test Li Wang via ltp
2026-01-29 10:08 ` Li Wang via ltp
2026-01-29 13:28 ` Petr Vorel
2026-01-29 13:44 ` Li Wang via ltp
2026-01-29 14:24 ` Cyril Hrubis
2026-01-30 0:01 ` Li Wang via ltp
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.