[LTP] [PATCH] mmap22: Bound child memory allocation to avoid OOM

[LTP] [PATCH] mmap22: Bound child memory allocation to avoid OOM

[Attila Fazekas via ltp]

Currently, stress_child() allocates memory in an infinite loop without
bounds. Under memory pressure, once the cgroup memory limit is reached
and the parent's droppable pages are reclaimed, the child continues
allocating non-droppable heap memory. This eventually exhausts the cgroup
limit and triggers the cgroup OOM killer, terminating either the child or
the test parent process.

Fix this by bounding the child's allocation to CHILD_ALLOC_MAX
(ALLOC_SIZE + 32MB). The total requested memory (128MB parent + 160MB
child = 288MB) exceeds the 256MB cgroup limit by 32MB, which reliably
forces the kernel to reclaim MAP_DROPPABLE pages. Once CHILD_ALLOC_MAX is
reached, the child calls pause() to keep memory resident. Since 160MB is
well below the 256MB cgroup limit, non-droppable memory alone cannot
trigger an OOM.

Assisted-by: Gemini CLI
Signed-off-by: Attila Fazekas <afazekas@redhat.com>
---
 testcases/kernel/syscalls/mmap/mmap22.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/testcases/kernel/syscalls/mmap/mmap22.c b/testcases/kernel/syscalls/mmap/mmap22.c
index 0b95c578f..16fcb6dd8 100644
--- a/testcases/kernel/syscalls/mmap/mmap22.c
+++ b/testcases/kernel/syscalls/mmap/mmap22.c
@@ -26,6 +26,7 @@
 
 #define MEM_LIMIT (256 * TST_MB)
 #define ALLOC_SIZE (128 * TST_MB)
+#define CHILD_ALLOC_MAX (ALLOC_SIZE + 32 * TST_MB)
 
 static struct tst_cg_group *cg_child;
 static pid_t child;
@@ -33,13 +34,15 @@ static int page_size;
 
 static void stress_child(void)
 {
-	for (;;) {
+	for (size_t sz = 0; sz < CHILD_ALLOC_MAX; sz += page_size) {
 		char *buf = malloc(page_size);
 
 		if (!buf)
-			exit(1);
+			break;
 		memset(buf, 'B', page_size);
 	}
+
+	pause();
 }
 
 static void test_mmap(void)
-- 
2.56.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] mmap22: Bound child memory allocation to avoid OOM

[linuxtestproject.agent]

Hi Attila,

On Mon Nov 2 11:57:04 2026 +0100, Attila Fazekas wrote:
> mmap22: Bound child memory allocation to avoid OOM

Verdict - Reviewed

---
Note:

The agent can sometimes produce false positives although often its
findings are genuine. If you find issues with the review, please
comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] mmap22: Bound child memory allocation to avoid OOM

[Petr Vorel]

Hi Attila,

> Currently, stress_child() allocates memory in an infinite loop without
> bounds. Under memory pressure, once the cgroup memory limit is reached
> and the parent's droppable pages are reclaimed, the child continues
> allocating non-droppable heap memory. This eventually exhausts the cgroup
> limit and triggers the cgroup OOM killer, terminating either the child or
> the test parent process.

> Fix this by bounding the child's allocation to CHILD_ALLOC_MAX
> (ALLOC_SIZE + 32MB). The total requested memory (128MB parent + 160MB
> child = 288MB) exceeds the 256MB cgroup limit by 32MB, which reliably
> forces the kernel to reclaim MAP_DROPPABLE pages. Once CHILD_ALLOC_MAX is
> reached, the child calls pause() to keep memory resident. Since 160MB is
> well below the 256MB cgroup limit, non-droppable memory alone cannot
> trigger an OOM.

LGTM, but it'd be nice if somebody else double check.
Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

> Assisted-by: Gemini CLI
> Signed-off-by: Attila Fazekas <afazekas@redhat.com>
> ---
>  testcases/kernel/syscalls/mmap/mmap22.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)

> diff --git a/testcases/kernel/syscalls/mmap/mmap22.c b/testcases/kernel/syscalls/mmap/mmap22.c
> index 0b95c578f..16fcb6dd8 100644
> --- a/testcases/kernel/syscalls/mmap/mmap22.c
> +++ b/testcases/kernel/syscalls/mmap/mmap22.c
> @@ -26,6 +26,7 @@

>  #define MEM_LIMIT (256 * TST_MB)
>  #define ALLOC_SIZE (128 * TST_MB)
> +#define CHILD_ALLOC_MAX (ALLOC_SIZE + 32 * TST_MB)

>  static struct tst_cg_group *cg_child;
>  static pid_t child;
> @@ -33,13 +34,15 @@ static int page_size;

>  static void stress_child(void)
>  {
> -	for (;;) {
> +	for (size_t sz = 0; sz < CHILD_ALLOC_MAX; sz += page_size) {
>  		char *buf = malloc(page_size);

>  		if (!buf)
> -			exit(1);
> +			break;
>  		memset(buf, 'B', page_size);
>  	}
> +
> +	pause();
>  }

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] mmap22: Bound child memory allocation to avoid OOM

[Cyril Hrubis]

Hi!
> Assisted-by: Gemini CLI
> Signed-off-by: Attila Fazekas <afazekas@redhat.com>
> ---
>  testcases/kernel/syscalls/mmap/mmap22.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/testcases/kernel/syscalls/mmap/mmap22.c b/testcases/kernel/syscalls/mmap/mmap22.c
> index 0b95c578f..16fcb6dd8 100644
> --- a/testcases/kernel/syscalls/mmap/mmap22.c
> +++ b/testcases/kernel/syscalls/mmap/mmap22.c
> @@ -26,6 +26,7 @@
>  
>  #define MEM_LIMIT (256 * TST_MB)
>  #define ALLOC_SIZE (128 * TST_MB)
> +#define CHILD_ALLOC_MAX (ALLOC_SIZE + 32 * TST_MB)
                             ^
			     Wouldn't this be better:
			     (MEM_LIMIT - ALLOC_SIZE + 32*TST_MB)

That way this would stil work if we change either MEM_LIMIT or
ALLOC_SIZE.

Otherwise the change looks good.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] mmap22: Bound child memory allocation to avoid OOM

[Attila Fazekas via ltp]

On Wed, Nov 4, 2026 at 5:41 PM Cyril Hrubis <chrubis@suse.cz> wrote:

> Hi!
> > Assisted-by: Gemini CLI
> > Signed-off-by: Attila Fazekas <afazekas@redhat.com>
> > ---
> >  testcases/kernel/syscalls/mmap/mmap22.c | 7 +++++--
> >  1 file changed, 5 insertions(+), 2 deletions(-)
> >
> > diff --git a/testcases/kernel/syscalls/mmap/mmap22.c
> b/testcases/kernel/syscalls/mmap/mmap22.c
> > index 0b95c578f..16fcb6dd8 100644
> > --- a/testcases/kernel/syscalls/mmap/mmap22.c
> > +++ b/testcases/kernel/syscalls/mmap/mmap22.c
> > @@ -26,6 +26,7 @@
> >
> >  #define MEM_LIMIT (256 * TST_MB)
> >  #define ALLOC_SIZE (128 * TST_MB)
> > +#define CHILD_ALLOC_MAX (ALLOC_SIZE + 32 * TST_MB)
>                              ^
>                              Wouldn't this be better:
>                              (MEM_LIMIT - ALLOC_SIZE + 32*TST_MB)
>
> This is better, I sent v2.
Thanks for the review.


> That way this would stil work if we change either MEM_LIMIT or
> ALLOC_SIZE.
>
> Otherwise the change looks good.
>
> --
> Cyril Hrubis
> chrubis@suse.cz
>
>

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] mmap22: Bound child memory allocation to avoid OOM

[linuxtestproject.agent]

Hi Attila,

On Thu Nov 5 09:02:45 2026 +0100, Attila Fazekas <afazekas@redhat.com> wrote:
> mmap22: Bound child memory allocation to avoid OOM

Verdict - Reviewed

---
Note:

The agent can sometimes produce false positives although often its
findings are genuine. If you find issues with the review, please
comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp