Re: [LTP] [PATCH] mm/oom05: new testcase

Jan Stancek
2012-07-03T08:03:18Z

> From: "Zhouping Liu" <zliu@redhat.com>
> To: gaowanlong@cn.fujitsu.com
> Cc: "LTP List" <ltp-list@lists.sourceforge.net>, "Jan Stancek" <jstancek@redhat.com>, "Caspar Zhang"
> <caspar@casparzhang.com>, "CAI Qian" <caiqian@redhat.com>
> Sent: Friday, 29 June, 2012 6:37:19 AM
> Subject: Re: [LTP] [PATCH] mm/oom05: new testcase
>
> Hi,
>
> ----- Original Message -----
> > From: "Wanlong Gao" <gaowanlong@cn.fujitsu.com>
> > To: "Zhouping Liu" <zliu@redhat.com>
> > Cc: "LTP List" <ltp-list@lists.sourceforge.net>, "Jan Stancek"
> > <jstancek@redhat.com>
> > Sent: Friday, June 29, 2012 10:39:57 AM
> > Subject: Re: [LTP] [PATCH] mm/oom05: new testcase
> >
> > On 06/29/2012 10:23 AM, Zhouping Liu wrote:
> > > Hi,
> > >
> > > ----- Original Message -----
> > >> From: "Zhouping Liu" <zliu@redhat.com>
> > >> To: "LTP List" <ltp-list@lists.sourceforge.net>
> > >> Sent: Tuesday, June 26, 2012 11:57:35 PM
> > >> Subject: [LTP] [PATCH] mm/oom05: new testcase
> > >>
> > >> add a new testcase to test OOM on each NUMA node
> > >
> > > I have tested it in several systems with 2 NUMA nodes and 8 NUMA
> > > nodes,
> > > they are all good, any comment?
> > >
> > > Thanks,
> > > Zhouping
> > >
> > >>
> > >> Signed-off-by: Zhouping Liu <zliu@redhat.com>
> > >> ---
> > >> runtest/mm | 1 +
> > >> testcases/kernel/mem/oom/oom05.c | 131
> > >> ++++++++++++++++++++++++++++++++++++++
> > >> 2 files changed, 132 insertions(+)
> > >> create mode 100644 testcases/kernel/mem/oom/oom05.c
> > >>
> > >> diff --git a/runtest/mm b/runtest/mm
> > >> index 2dd66f0..a60c04b 100644
> > >> --- a/runtest/mm
> > >> +++ b/runtest/mm
> > >> @@ -77,6 +77,7 @@ oom01 oom01
> > >> oom02 oom02
> > >> oom03 oom03
> > >> oom04 oom04
> > >> +oom05 oom05
> > >>
> > >> swapping01 swapping01 -i 5
> > >>
> > >> diff --git a/testcases/kernel/mem/oom/oom05.c
> > >> b/testcases/kernel/mem/oom/oom05.c
> > >> new file mode 100644
> > >> index 0000000..046ddbb
> > >> --- /dev/null
> > >> +++ b/testcases/kernel/mem/oom/oom05.c
> > >> @@ -0,0 +1,131 @@
> > >> +/*
> > >> + * Out Of Memory (OOM) on Each NUMA nodes
> > >> + *
> > >> + * The program is designed to cope with unpredictable like
> > >> amount
> > >> and
> > >> + * system physical memory, swap size and other VMM technology
> > >> like
> > >> KSM,
> > >> + * memcg, memory hotplug and so on which may affect the OOM
> > >> + * behaviours. It simply increase the memory consumption 3G
> > >> each
> > >> time
> > >> + * until all the available memory is consumed and OOM is
> > >> triggered.
> > >> + *
> > >> + * Copyright (C) 2012 Red Hat, Inc.
> > >> + * This program is free software; you can redistribute it
> > >> and/or
> > >> + * modify it under the terms of version 2 of the GNU General
> > >> Public
> > >> + * License as published by the Free Software Foundation.
> > >> + *
> > >> + * This program is distributed in the hope that it would be
> > >> useful,
> > >> + * but WITHOUT ANY WARRANTY; without even the implied warranty
> > >> of
> > >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > >> + *
> > >> + * Further, this software is distributed without any warranty
> > >> that
> > >> it
> > >> + * is free of the rightful claim of any third person regarding
> > >> + * infringement or the like. Any license provided herein,
> > >> whether
> > >> + * implied or otherwise, applies only to this software file.
> > >> Patent
> > >> + * licenses, if any, provided herein do not apply to
> > >> combinations
> > >> of
> > >> + * this program with other software, or any other product
> > >> whatsoever.
> > >> + *
> > >> + * You should have received a copy of the GNU General Public
> > >> License
> > >> + * along with this program; if not, write the Free Software
> > >> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
> > >> MA
> > >> + * 02110-1301, USA.
> > >> + */
> > >> +#include <sys/types.h>
> > >> +#include <sys/stat.h>
> > >> +#include <sys/wait.h>
> > >> +#include <errno.h>
> > >> +#include <fcntl.h>
> > >> +#include <stdio.h>
> > >> +#include <stdlib.h>
> > >> +#include <unistd.h>
> > >> +#include "test.h"
> > >> +#include "safe_macros.h"
> > >> +#include "usctest.h"
> > >> +#include "mem.h"
> > >> +
> > >> +char *TCID = "oom05";
> > >> +int TST_TOTAL = 1;
> > >> +
> > >> +#if HAVE_NUMA_H && HAVE_LINUX_MEMPOLICY_H && HAVE_NUMAIF_H \
> > >> + && HAVE_MPOL_CONSTANTS
> > >> +#include <numa.h>
> > >> +#include <numaif.h>
> > >> +
> > >> +int main(int argc, char *argv[])
> > >> +{
> > >> + char *msg;
> > >> + int lc;
> > >> + long nodes[MAXNODES];
> > >> + int nnodes, i;
> > >> + pid_t pids[MAXNODES];
> > >> + int status;
> > >> + unsigned long nmasks[MAXNODES];
> > >> +
> > >> + msg = parse_opts(argc, argv, NULL, NULL);
> > >> + if (msg != NULL)
> > >> + tst_brkm(TBROK, NULL, "OPTION PARSING ERROR - %s", msg);
> > >> +
> > >> +#if __WORDSIZE == 32
> > >> + tst_brkm(TCONF, NULL, "test is not designed for 32-bit
> > >> system.");
> > >> +#endif
> > >> + nnodes = count_numa(nodes);
> > >> + if (nnodes <= 1)
> > >> + tst_brkm(TCONF, NULL, "required a NUMA system.");
> > >> +
> > >> + setup();
> > >> +
> > >> + set_sys_tune("overcommit_memory", 1, 1);
> > >> +
> > >> + for (lc = 0; TEST_LOOPING(lc); lc++) {
> > >> + Tst_count = 0;
> > >> +
> > >> + for (i = 0; i < nnodes; i++) {
> > >> + nmasks[i] = 1 << nodes[i];
> > >> + switch (pids[i] = fork()) {
> > >> + case -1:
> > >> + tst_brkm(TBROK|TERRNO, cleanup, "fork");
> > >> + case 0:
> > >> + if (set_mempolicy(MPOL_BIND, &nmasks[i],
> > >> + MAXNODES) == -1)
> > >> + tst_brkm(TBROK|TERRNO, cleanup,
> > >> + "set_mempolicy");
> > >> + tst_resm(TINFO, "start oom on numa node%lu",
> > >> + nodes[i]);
> > >> + oom(NORMAL, 0, 0);
> > >> + exit(0);
> > >> + }
> > >> + }
> >
> > Hi Zhouping,
> >
> > Does it need get_allowed_nodes() that Jan added?
> >
>
> yes, I met some errors when the system has unbalance NUMA system:
>
> the follow test machine has 2 numa nodes 0-1, I offline all node1's
> cpu
> # echo 0 > /sys/devices/system/node1/cpu*/online
>
> then you will find node1 don't contain any available cpus:
>
> # numactl -H
> available: 2 nodes (0-1)
> node 0 cpus: 0 1 2 3
> node 0 size: 4084 MB
> node 0 free: 2929 MB
> node 1 cpus:
> node 1 size: 4096 MB
> node 1 free: 3949 MB
> node distances:
> node 0 1
> 0: 10 21
> 1: 21 10
>
> under such above setup, oom[2|4] will failed:
>
> ...
> oom02 0 TINFO : process cpuset.
> oom02 1 TBROK : write /dev/cpuset/1/cpuset.cpus:
> errno=EINVAL(22): Invalid argument
> oom02 2 TBROK : Remaining cases broken
> oom02 0 TINFO : set overcommit_memory to 0
> oom02 0 TWARN : umount /dev/cpuset: errno=EBUSY(16): Device
> or resource busy
> oom02 0 TWARN : rmdir /dev/cpuset: errno=EBUSY(16): Device
> or resource busy
>
> the failure is because node1 didn't have available cpus, but it tried
> to write node1's cpus to /dev/cpuset/1/cpuset.cpus.
>
> and I tested get_allowed_nodes(), it didn't fix the error,
> with above setup, I expected get_allowed_nodes_arr() can mark node1
> is a unavailable node,
> but in actual it didn't, and it's the similar with count_numa()
> inside testcase/kernel/mem/lib/mem.c.
> so, Jan, do you have any thoughts to avoid the issues? if
> get-allowed_nodes* can fix these issues, I also suggest we can
> define them in ltp/include/numa_helper.h or other file name,
> comments?

Zhouping,

Are other tests, such as move_pages also failing on this setup?

Regards,
Jan

>
> Thansk,
> Zhuoping