Re: [LTP] [PATCH] mm/oom05: new testcase

Jan Stancek
2012-07-10T10:50:09Z

> From: "Caspar Zhang" <caspar@casparzhang.com>
> To: "Zhouping Liu" <zliu@redhat.com>
> Cc: gaowanlong@cn.fujitsu.com, "LTP List" <ltp-list@lists.sourceforge.net>, "Jan Stancek" <jstancek@redhat.com>, "CAI
> Qian" <caiqian@redhat.com>
> Sent: Tuesday, 10 July, 2012 12:37:10 PM
> Subject: Re: [LTP] [PATCH] mm/oom05: new testcase
>
> On 06/29/2012 12:37 PM, Zhouping Liu wrote:
> > Hi,
> >
> > ----- Original Message -----
> >> From: "Wanlong Gao" <gaowanlong@cn.fujitsu.com>
> >> To: "Zhouping Liu" <zliu@redhat.com>
> >> Cc: "LTP List" <ltp-list@lists.sourceforge.net>, "Jan Stancek"
> >> <jstancek@redhat.com>
> >> Sent: Friday, June 29, 2012 10:39:57 AM
> >> Subject: Re: [LTP] [PATCH] mm/oom05: new testcase
> >>
> >> On 06/29/2012 10:23 AM, Zhouping Liu wrote:
> >>> Hi,
> >>>
> >>> ----- Original Message -----
> >>>> From: "Zhouping Liu" <zliu@redhat.com>
> >>>> To: "LTP List" <ltp-list@lists.sourceforge.net>
> >>>> Sent: Tuesday, June 26, 2012 11:57:35 PM
> >>>> Subject: [LTP] [PATCH] mm/oom05: new testcase
> >>>>
> >>>> add a new testcase to test OOM on each NUMA node
> >>>
> >>> I have tested it in several systems with 2 NUMA nodes and 8 NUMA
> >>> nodes,
> >>> they are all good, any comment?
> >>>
> >>> Thanks,
> >>> Zhouping
> >>>
> >>>>
> >>>> Signed-off-by: Zhouping Liu <zliu@redhat.com>
> >>>> ---
> >>>> runtest/mm | 1 +
> >>>> testcases/kernel/mem/oom/oom05.c | 131
> >>>> ++++++++++++++++++++++++++++++++++++++
> >>>> 2 files changed, 132 insertions(+)
> >>>> create mode 100644 testcases/kernel/mem/oom/oom05.c
> >>>>
> >>>> diff --git a/runtest/mm b/runtest/mm
> >>>> index 2dd66f0..a60c04b 100644
> >>>> --- a/runtest/mm
> >>>> +++ b/runtest/mm
> >>>> @@ -77,6 +77,7 @@ oom01 oom01
> >>>> oom02 oom02
> >>>> oom03 oom03
> >>>> oom04 oom04
> >>>> +oom05 oom05
> >>>>
> >>>> swapping01 swapping01 -i 5
> >>>>
> >>>> diff --git a/testcases/kernel/mem/oom/oom05.c
> >>>> b/testcases/kernel/mem/oom/oom05.c
> >>>> new file mode 100644
> >>>> index 0000000..046ddbb
> >>>> --- /dev/null
> >>>> +++ b/testcases/kernel/mem/oom/oom05.c
> >>>> @@ -0,0 +1,131 @@
> >>>> +/*
> >>>> + * Out Of Memory (OOM) on Each NUMA nodes
> >>>> + *
> >>>> + * The program is designed to cope with unpredictable like
> >>>> amount
> >>>> and
> >>>> + * system physical memory, swap size and other VMM technology
> >>>> like
> >>>> KSM,
> >>>> + * memcg, memory hotplug and so on which may affect the OOM
> >>>> + * behaviours. It simply increase the memory consumption 3G
> >>>> each
> >>>> time
> >>>> + * until all the available memory is consumed and OOM is
> >>>> triggered.
> >>>> + *
> >>>> + * Copyright (C) 2012 Red Hat, Inc.
> >>>> + * This program is free software; you can redistribute it
> >>>> and/or
> >>>> + * modify it under the terms of version 2 of the GNU General
> >>>> Public
> >>>> + * License as published by the Free Software Foundation.
> >>>> + *
> >>>> + * This program is distributed in the hope that it would be
> >>>> useful,
> >>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty
> >>>> of
> >>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> >>>> + *
> >>>> + * Further, this software is distributed without any warranty
> >>>> that
> >>>> it
> >>>> + * is free of the rightful claim of any third person regarding
> >>>> + * infringement or the like. Any license provided herein,
> >>>> whether
> >>>> + * implied or otherwise, applies only to this software file.
> >>>> Patent
> >>>> + * licenses, if any, provided herein do not apply to
> >>>> combinations
> >>>> of
> >>>> + * this program with other software, or any other product
> >>>> whatsoever.
> >>>> + *
> >>>> + * You should have received a copy of the GNU General Public
> >>>> License
> >>>> + * along with this program; if not, write the Free Software
> >>>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
> >>>> MA
> >>>> + * 02110-1301, USA.
> >>>> + */
> >>>> +#include <sys/types.h>
> >>>> +#include <sys/stat.h>
> >>>> +#include <sys/wait.h>
> >>>> +#include <errno.h>
> >>>> +#include <fcntl.h>
> >>>> +#include <stdio.h>
> >>>> +#include <stdlib.h>
> >>>> +#include <unistd.h>
> >>>> +#include "test.h"
> >>>> +#include "safe_macros.h"
> >>>> +#include "usctest.h"
> >>>> +#include "mem.h"
> >>>> +
> >>>> +char *TCID = "oom05";
> >>>> +int TST_TOTAL = 1;
> >>>> +
> >>>> +#if HAVE_NUMA_H && HAVE_LINUX_MEMPOLICY_H && HAVE_NUMAIF_H \
> >>>> + && HAVE_MPOL_CONSTANTS
> >>>> +#include <numa.h>
> >>>> +#include <numaif.h>
> >>>> +
> >>>> +int main(int argc, char *argv[])
> >>>> +{
> >>>> + char *msg;
> >>>> + int lc;
> >>>> + long nodes[MAXNODES];
> >>>> + int nnodes, i;
> >>>> + pid_t pids[MAXNODES];
> >>>> + int status;
> >>>> + unsigned long nmasks[MAXNODES];
> >>>> +
> >>>> + msg = parse_opts(argc, argv, NULL, NULL);
> >>>> + if (msg != NULL)
> >>>> + tst_brkm(TBROK, NULL, "OPTION PARSING ERROR - %s", msg);
> >>>> +
> >>>> +#if __WORDSIZE == 32
> >>>> + tst_brkm(TCONF, NULL, "test is not designed for 32-bit
> >>>> system.");
> >>>> +#endif
> >>>> + nnodes = count_numa(nodes);
> >>>> + if (nnodes <= 1)
> >>>> + tst_brkm(TCONF, NULL, "required a NUMA system.");
> >>>> +
> >>>> + setup();
> >>>> +
> >>>> + set_sys_tune("overcommit_memory", 1, 1);
> >>>> +
> >>>> + for (lc = 0; TEST_LOOPING(lc); lc++) {
> >>>> + Tst_count = 0;
> >>>> +
> >>>> + for (i = 0; i < nnodes; i++) {
> >>>> + nmasks[i] = 1 << nodes[i];
> >>>> + switch (pids[i] = fork()) {
> >>>> + case -1:
> >>>> + tst_brkm(TBROK|TERRNO, cleanup, "fork");
> >>>> + case 0:
> >>>> + if (set_mempolicy(MPOL_BIND, &nmasks[i],
> >>>> + MAXNODES) == -1)
> >>>> + tst_brkm(TBROK|TERRNO, cleanup,
> >>>> + "set_mempolicy");
> >>>> + tst_resm(TINFO, "start oom on numa node%lu",
> >>>> + nodes[i]);
> >>>> + oom(NORMAL, 0, 0);
> >>>> + exit(0);
> >>>> + }
> >>>> + }
> >>
> >> Hi Zhouping,
> >>
> >> Does it need get_allowed_nodes() that Jan added?
> >>
> >
> > yes, I met some errors when the system has unbalance NUMA system:
> >
> > the follow test machine has 2 numa nodes 0-1, I offline all node1's
> > cpu
> > # echo 0 > /sys/devices/system/node1/cpu*/online
> >
> > then you will find node1 don't contain any available cpus:
> >
> > # numactl -H
> > available: 2 nodes (0-1)
> > node 0 cpus: 0 1 2 3
> > node 0 size: 4084 MB
> > node 0 free: 2929 MB
> > node 1 cpus:
> > node 1 size: 4096 MB
> > node 1 free: 3949 MB
> > node distances:
> > node 0 1
> > 0: 10 21
> > 1: 21 10
> >
> > under such above setup, oom[2|4] will failed:
> >
> > ...
> > oom02 0 TINFO : process cpuset.
> > oom02 1 TBROK : write /dev/cpuset/1/cpuset.cpus:
> > errno=EINVAL(22): Invalid argument
> > oom02 2 TBROK : Remaining cases broken
> > oom02 0 TINFO : set overcommit_memory to 0
> > oom02 0 TWARN : umount /dev/cpuset: errno=EBUSY(16):
> > Device or resource busy
> > oom02 0 TWARN : rmdir /dev/cpuset: errno=EBUSY(16): Device
> > or resource busy
> >
> > the failure is because node1 didn't have available cpus, but it
> > tried to write node1's cpus to /dev/cpuset/1/cpuset.cpus.
> >
> > and I tested get_allowed_nodes(), it didn't fix the error,
> > with above setup, I expected get_allowed_nodes_arr() can mark node1
> > is a unavailable node,
> > but in actual it didn't, and it's the similar with count_numa()
> > inside testcase/kernel/mem/lib/mem.c.
> > so, Jan, do you have any thoughts to avoid the issues? if
> > get-allowed_nodes* can fix these issues, I also suggest we can
> > define them in ltp/include/numa_helper.h or other file name,
> > comments?
>
> Looks like get_allowed_nodes* functions only check allowed memories,
> but
> doesn't check cpus, maybe adding a numa_node_to_cpus() check can
> solve this?

I was thinking the same:
http://article.gmane.org/gmane.linux.ltp/16348

but considering, if we apply that everywhere or introduce some kind of flag,
so the test could specify what nodes it is interested in (memory/cpu/both).

Regards,
Jan

>
> Thanks,
> Caspar