* [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes
@ 2019-05-08 8:44 Li Wang
2019-05-08 8:44 ` [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id Li Wang
2019-05-09 7:16 ` [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes Balamuruhan S
0 siblings, 2 replies; 13+ messages in thread
From: Li Wang @ 2019-05-08 8:44 UTC (permalink / raw)
To: ltp
Numa test failed on such machine which has non-continuous numa nodes,
it gets wrong data because of the below syntax rule is not applicable
to that special situation.
` numastat -p $pid |awk '/^Total/ {print $'$((node+2))'}'
In this patch, we fix that via a bit complex way of awk to get the
allocated memory in specified node.
# numactl -H
available: 2 nodes (0,8)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63
node 0 size: 257741 MB
node 0 free: 253158 MB
node 8 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
96 97 98 99 100 101 102 103 104 105 106 107 108
109 110 111 112 113 114 115 116 117 118 119 120
121 122 123 124 125 126 127
node 8 size: 261752 MB
node 8 free: 240933 MB
node distances:
node 0 8
0: 10 40
8: 40 10
# numastat -p $pid
Per-node process memory usage (in MBs) for PID 34168 (support_numa)
Node 0 Node 8 Total
--------------- --------------- ---------------
Huge 0.00 0.00 0.00
Heap 0.00 0.12 0.12
Stack 0.00 0.06 0.06
Private 1.62 1.50 3.12
---------------- --------------- --------------- ---------------
Total 1.62 1.69 3.31
Signed-off-by: Li Wang <liwang@redhat.com>
Cc: Chunyu Hu <chuhu@redhat.com>
Cc: Cyril Hrubis <chrubis@suse.cz>
---
testcases/kernel/numa/numa01.sh | 26 ++++++++++++++++++++++----
1 file changed, 22 insertions(+), 4 deletions(-)
diff --git a/testcases/kernel/numa/numa01.sh b/testcases/kernel/numa/numa01.sh
index 33393ac8d..7f3dee849 100755
--- a/testcases/kernel/numa/numa01.sh
+++ b/testcases/kernel/numa/numa01.sh
@@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat"
extract_numastat_p()
{
local pid=$1
- local node=$(($2 + 2))
-
- echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}')
+ local node=$2
+
+ echo $(numastat -p $pid | \
+ awk -v node=$node '/Node/ { \
+ gsub("Node", ""); \
+ for (i=0; i<NF; i++) \
+ if ($i == node) \
+ col=i+1; \
+ next \
+ } \
+ /^Total/ {print $col}' \
+ )
}
check_for_support_numa()
@@ -363,7 +372,16 @@ test9()
pid=$!
TST_RETRY_FUNC "check_for_support_numa $pid" 0
- Mem_huge=$(echo $(numastat -p $pid |awk '/^Huge/ {print $'$((node+2))'}'))
+ Mem_huge=$(echo $(numastat -p $pid | \
+ awk -v node=$node '/Node/ { \
+ gsub("Node", ""); \
+ for (i=0; i<NF; i++) \
+ if ($i == node) \
+ col=i+1; \
+ next \
+ } \
+ /^Huge/ {print $col}') \
+ )
Mem_huge=$((${Mem_huge%.*} * 1024))
if [ "$Mem_huge" -lt "$HPAGE_SIZE" ]; then
--
2.20.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id 2019-05-08 8:44 [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes Li Wang @ 2019-05-08 8:44 ` Li Wang 2019-05-09 6:45 ` Balamuruhan S 2019-05-09 7:16 ` [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes Balamuruhan S 1 sibling, 1 reply; 13+ messages in thread From: Li Wang @ 2019-05-08 8:44 UTC (permalink / raw) To: ltp For some ppc64le systems, it has non-continuous numa nodes in hardware configuration. So we're hitting the below warnings while running set_mempolicy tests on that. To fix this issue, let's just remove restrictions on numa node-id in get_mempolicy(). Error Log --------- tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s tst_numa.c:190: INFO: Found 2 NUMA memory nodes set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 tst_numa.c:26: INFO: Node 0 allocated 16 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 ... tst_numa.c:26: INFO: Node 0 allocated 0 pages tst_numa.c:26: INFO: Node 8 allocated 0 pages set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16 lscpu ----- Architecture: ppc64le ... CPU(s): 128 Core(s) per socket: 16 Socket(s): 2 NUMA node(s): 2 Model name: POWER9, altivec supported ... NUMA node0 CPU(s): 0-63 NUMA node8 CPU(s): 64-127 Signed-off-by: Li Wang <liwang@redhat.com> Cc: Cyril Hrubis <chrubis@suse.cz> --- libs/libltpnuma/tst_numa.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libs/libltpnuma/tst_numa.c b/libs/libltpnuma/tst_numa.c index 0ba6daf39..56c8640ff 100644 --- a/libs/libltpnuma/tst_numa.c +++ b/libs/libltpnuma/tst_numa.c @@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes, if (ret < 0) tst_brk(TBROK | TERRNO, "get_mempolicy() failed"); - if (node < 0 || (unsigned int)node >= nodes->cnt) { - tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node); + if (node < 0) { + tst_res(TWARN, + "get_mempolicy(...) returned invalid node %i\n", node); continue; } -- 2.20.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id 2019-05-08 8:44 ` [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id Li Wang @ 2019-05-09 6:45 ` Balamuruhan S 2019-05-09 8:00 ` Li Wang ` (2 more replies) 0 siblings, 3 replies; 13+ messages in thread From: Balamuruhan S @ 2019-05-09 6:45 UTC (permalink / raw) To: ltp On Wed, May 08, 2019 at 04:44:47PM +0800, Li Wang wrote: > For some ppc64le systems, it has non-continuous numa nodes in > hardware configuration. So we're hitting the below warnings while > running set_mempolicy tests on that. To fix this issue, let's just > remove restrictions on numa node-id in get_mempolicy(). > > Error Log > --------- > tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s > tst_numa.c:190: INFO: Found 2 NUMA memory nodes > set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 > tst_numa.c:26: INFO: Node 0 allocated 16 pages > tst_numa.c:26: INFO: Node 8 allocated 0 pages > set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 > set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > ... > tst_numa.c:26: INFO: Node 0 allocated 0 pages > tst_numa.c:26: INFO: Node 8 allocated 0 pages > set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16 > > lscpu > ----- > Architecture: ppc64le > ... > CPU(s): 128 > Core(s) per socket: 16 > Socket(s): 2 > NUMA node(s): 2 > Model name: POWER9, altivec supported > ... > NUMA node0 CPU(s): 0-63 > NUMA node8 CPU(s): 64-127 > > Signed-off-by: Li Wang <liwang@redhat.com> > Cc: Cyril Hrubis <chrubis@suse.cz> > --- > libs/libltpnuma/tst_numa.c | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/libs/libltpnuma/tst_numa.c b/libs/libltpnuma/tst_numa.c > index 0ba6daf39..56c8640ff 100644 > --- a/libs/libltpnuma/tst_numa.c > +++ b/libs/libltpnuma/tst_numa.c > @@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes, > if (ret < 0) > tst_brk(TBROK | TERRNO, "get_mempolicy() failed"); > > - if (node < 0 || (unsigned int)node >= nodes->cnt) { > - tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node); > + if (node < 0) { It is better to have check for powerpc and remove restriction as it will still hold good for other arch. -- Bala > + tst_res(TWARN, > + "get_mempolicy(...) returned invalid node %i\n", node); > continue; > } > > -- > 2.20.1 > > > -- > Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id 2019-05-09 6:45 ` Balamuruhan S @ 2019-05-09 8:00 ` Li Wang 2019-05-09 9:23 ` Balamuruhan S 2019-05-09 15:50 ` Jan Stancek 2019-05-14 14:30 ` Cyril Hrubis 2 siblings, 1 reply; 13+ messages in thread From: Li Wang @ 2019-05-09 8:00 UTC (permalink / raw) To: ltp On Thu, May 9, 2019 at 2:45 PM Balamuruhan S <bala24@linux.vnet.ibm.com> wrote: > On Wed, May 08, 2019 at 04:44:47PM +0800, Li Wang wrote: > > For some ppc64le systems, it has non-continuous numa nodes in > > hardware configuration. So we're hitting the below warnings while > > running set_mempolicy tests on that. To fix this issue, let's just > > remove restrictions on numa node-id in get_mempolicy(). > > > > Error Log > > --------- > > tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s > > tst_numa.c:190: INFO: Found 2 NUMA memory nodes > > set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 > > tst_numa.c:26: INFO: Node 0 allocated 16 pages > > tst_numa.c:26: INFO: Node 8 allocated 0 pages > > set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 > > set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > ... > > tst_numa.c:26: INFO: Node 0 allocated 0 pages > > tst_numa.c:26: INFO: Node 8 allocated 0 pages > > set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16 > > > > lscpu > > ----- > > Architecture: ppc64le > > ... > > CPU(s): 128 > > Core(s) per socket: 16 > > Socket(s): 2 > > NUMA node(s): 2 > > Model name: POWER9, altivec supported > > ... > > NUMA node0 CPU(s): 0-63 > > NUMA node8 CPU(s): 64-127 > > > > Signed-off-by: Li Wang <liwang@redhat.com> > > Cc: Cyril Hrubis <chrubis@suse.cz> > > --- > > libs/libltpnuma/tst_numa.c | 5 +++-- > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > diff --git a/libs/libltpnuma/tst_numa.c b/libs/libltpnuma/tst_numa.c > > index 0ba6daf39..56c8640ff 100644 > > --- a/libs/libltpnuma/tst_numa.c > > +++ b/libs/libltpnuma/tst_numa.c > > @@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes, > > if (ret < 0) > > tst_brk(TBROK | TERRNO, "get_mempolicy() failed"); > > > > - if (node < 0 || (unsigned int)node >= nodes->cnt) { > > - tst_res(TWARN, "get_mempolicy(...) returned > invalid node %i\n", node); > > + if (node < 0) { > > It is better to have check for powerpc and remove restriction as it will > still hold good for other arch. > If the non-continuous numa nodes only appear in ppc machine, that should be necessary to check arch before removing the restriction. then the patch probably like: --- a/libs/libltpnuma/tst_numa.c +++ b/libs/libltpnuma/tst_numa.c @@ -88,7 +88,11 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes, if (ret < 0) tst_brk(TBROK | TERRNO, "get_mempolicy() failed"); +#if defined (__powerpc64__) || (__powerpc__) + if (node < 0) { +#elif if (node < 0 || (unsigned int)node >= nodes->cnt) { +#endif tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node); continue; } -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20190509/e93d1f8f/attachment-0001.html> ^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id 2019-05-09 8:00 ` Li Wang @ 2019-05-09 9:23 ` Balamuruhan S 0 siblings, 0 replies; 13+ messages in thread From: Balamuruhan S @ 2019-05-09 9:23 UTC (permalink / raw) To: ltp On Thu, May 09, 2019 at 04:00:40PM +0800, Li Wang wrote: > On Thu, May 9, 2019 at 2:45 PM Balamuruhan S <bala24@linux.vnet.ibm.com> > wrote: > > > On Wed, May 08, 2019 at 04:44:47PM +0800, Li Wang wrote: > > > For some ppc64le systems, it has non-continuous numa nodes in > > > hardware configuration. So we're hitting the below warnings while > > > running set_mempolicy tests on that. To fix this issue, let's just > > > remove restrictions on numa node-id in get_mempolicy(). > > > > > > Error Log > > > --------- > > > tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s > > > tst_numa.c:190: INFO: Found 2 NUMA memory nodes > > > set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 > > > tst_numa.c:26: INFO: Node 0 allocated 16 pages > > > tst_numa.c:26: INFO: Node 8 allocated 0 pages > > > set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 > > > set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 > > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > > ... > > > tst_numa.c:26: INFO: Node 0 allocated 0 pages > > > tst_numa.c:26: INFO: Node 8 allocated 0 pages > > > set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16 > > > > > > lscpu > > > ----- > > > Architecture: ppc64le > > > ... > > > CPU(s): 128 > > > Core(s) per socket: 16 > > > Socket(s): 2 > > > NUMA node(s): 2 > > > Model name: POWER9, altivec supported > > > ... > > > NUMA node0 CPU(s): 0-63 > > > NUMA node8 CPU(s): 64-127 > > > > > > Signed-off-by: Li Wang <liwang@redhat.com> > > > Cc: Cyril Hrubis <chrubis@suse.cz> > > > --- > > > libs/libltpnuma/tst_numa.c | 5 +++-- > > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > > > diff --git a/libs/libltpnuma/tst_numa.c b/libs/libltpnuma/tst_numa.c > > > index 0ba6daf39..56c8640ff 100644 > > > --- a/libs/libltpnuma/tst_numa.c > > > +++ b/libs/libltpnuma/tst_numa.c > > > @@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes, > > > if (ret < 0) > > > tst_brk(TBROK | TERRNO, "get_mempolicy() failed"); > > > > > > - if (node < 0 || (unsigned int)node >= nodes->cnt) { > > > - tst_res(TWARN, "get_mempolicy(...) returned > > invalid node %i\n", node); > > > + if (node < 0) { > > > > It is better to have check for powerpc and remove restriction as it will > > still hold good for other arch. > > > > If the non-continuous numa nodes only appear in ppc machine, that should be > necessary to check arch before removing the restriction. > > then the patch probably like: > > --- a/libs/libltpnuma/tst_numa.c > +++ b/libs/libltpnuma/tst_numa.c > @@ -88,7 +88,11 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes, > if (ret < 0) > tst_brk(TBROK | TERRNO, "get_mempolicy() failed"); > > +#if defined (__powerpc64__) || (__powerpc__) > + if (node < 0) { > +#elif > if (node < 0 || (unsigned int)node >= nodes->cnt) { > +#endif Looks good to me. Thanks! -- Bala > tst_res(TWARN, "get_mempolicy(...) returned invalid > node %i\n", node); > continue; > } > > > -- > Regards, > Li Wang ^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id 2019-05-09 6:45 ` Balamuruhan S 2019-05-09 8:00 ` Li Wang @ 2019-05-09 15:50 ` Jan Stancek 2019-05-14 14:30 ` Cyril Hrubis 2 siblings, 0 replies; 13+ messages in thread From: Jan Stancek @ 2019-05-09 15:50 UTC (permalink / raw) To: ltp ----- Original Message ----- > On Wed, May 08, 2019 at 04:44:47PM +0800, Li Wang wrote: > > For some ppc64le systems, it has non-continuous numa nodes in > > hardware configuration. So we're hitting the below warnings while > > running set_mempolicy tests on that. To fix this issue, let's just > > remove restrictions on numa node-id in get_mempolicy(). > > > > Error Log > > --------- > > tst_test.c:1096: INFO: Timeout per run is 0h 50m 00s > > tst_numa.c:190: INFO: Found 2 NUMA memory nodes > > set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 0 > > tst_numa.c:26: INFO: Node 0 allocated 16 pages > > tst_numa.c:26: INFO: Node 8 allocated 0 pages > > set_mempolicy01.c:82: PASS: child: Node 0 allocated 16 > > set_mempolicy01.c:63: PASS: set_mempolicy(MPOL_BIND) node 8 > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 8 > > ... > > tst_numa.c:26: INFO: Node 0 allocated 0 pages > > tst_numa.c:26: INFO: Node 8 allocated 0 pages > > set_mempolicy01.c:86: FAIL: child: Node 8 allocated 0, expected 16 > > > > lscpu > > ----- > > Architecture: ppc64le > > ... > > CPU(s): 128 > > Core(s) per socket: 16 > > Socket(s): 2 > > NUMA node(s): 2 > > Model name: POWER9, altivec supported > > ... > > NUMA node0 CPU(s): 0-63 > > NUMA node8 CPU(s): 64-127 > > > > Signed-off-by: Li Wang <liwang@redhat.com> > > Cc: Cyril Hrubis <chrubis@suse.cz> > > --- > > libs/libltpnuma/tst_numa.c | 5 +++-- > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > diff --git a/libs/libltpnuma/tst_numa.c b/libs/libltpnuma/tst_numa.c > > index 0ba6daf39..56c8640ff 100644 > > --- a/libs/libltpnuma/tst_numa.c > > +++ b/libs/libltpnuma/tst_numa.c > > @@ -88,8 +88,9 @@ void tst_nodemap_count_pages(struct tst_nodemap *nodes, > > if (ret < 0) > > tst_brk(TBROK | TERRNO, "get_mempolicy() failed"); > > > > - if (node < 0 || (unsigned int)node >= nodes->cnt) { > > - tst_res(TWARN, "get_mempolicy(...) returned invalid node %i\n", node); > > + if (node < 0) { > > It is better to have check for powerpc and remove restriction as it will > still hold good for other arch. Unless memory gets offlined or test harness restricts LTP to run only on some nodes (via numactl). Here's example of former: # uname -m x86_64 # numactl -H available: 8 nodes (0-7) node 0 cpus: 0 4 8 12 node 0 size: 4061 MB node 0 free: 3437 MB node 1 cpus: 16 20 24 28 node 1 size: 0 MB node 1 free: 0 MB node 2 cpus: 1 5 9 13 node 2 size: 4096 MB node 2 free: 3871 MB node 3 cpus: 17 21 25 29 node 3 size: 0 MB node 3 free: 0 MB node 4 cpus: 2 6 10 14 node 4 size: 4096 MB node 4 free: 3568 MB node 5 cpus: 18 22 26 30 node 5 size: 0 MB node 5 free: 0 MB node 6 cpus: 19 23 27 31 node 6 size: 0 MB node 6 free: 0 MB node 7 cpus: 3 7 11 15 node 7 size: 4095 MB node 7 free: 3802 MB tst_test.c:1096: INFO: Timeout per run is 0h 05m 00s tst_numa.c:190: INFO: Found 4 NUMA memory nodes ... tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 4 tst_numa.c:92: WARN: get_mempolicy(...) returned invalid node 7 > > -- Bala > > > + tst_res(TWARN, > > + "get_mempolicy(...) returned invalid node %i\n", node); > > continue; > > } > > > > -- > > 2.20.1 > > > > > > -- > > Mailing list info: https://lists.linux.it/listinfo/ltp > > > -- > Mailing list info: https://lists.linux.it/listinfo/ltp > ^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id 2019-05-09 6:45 ` Balamuruhan S 2019-05-09 8:00 ` Li Wang 2019-05-09 15:50 ` Jan Stancek @ 2019-05-14 14:30 ` Cyril Hrubis 2 siblings, 0 replies; 13+ messages in thread From: Cyril Hrubis @ 2019-05-14 14:30 UTC (permalink / raw) To: ltp Hi! > It is better to have check for powerpc and remove restriction as it will > still hold good for other arch. I would like to avoid adding arch specific ifdefs to the code, so for the release I've commited the original fix and we can add better checks later on. As I wrote checking that the numa id is <= than maximal used id would be much better and would work in all cases. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes 2019-05-08 8:44 [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes Li Wang 2019-05-08 8:44 ` [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id Li Wang @ 2019-05-09 7:16 ` Balamuruhan S 2019-05-09 7:54 ` Li Wang 1 sibling, 1 reply; 13+ messages in thread From: Balamuruhan S @ 2019-05-09 7:16 UTC (permalink / raw) To: ltp On Wed, May 08, 2019 at 04:44:46PM +0800, Li Wang wrote: > Numa test failed on such machine which has non-continuous numa nodes, > it gets wrong data because of the below syntax rule is not applicable > to that special situation. > ` numastat -p $pid |awk '/^Total/ {print $'$((node+2))'}' > In this patch, we fix that via a bit complex way of awk to get the > allocated memory in specified node. > > # numactl -H > available: 2 nodes (0,8) > node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 > 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 > 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 > 51 52 53 54 55 56 57 58 59 60 61 62 63 > node 0 size: 257741 MB > node 0 free: 253158 MB > node 8 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 > 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 > 96 97 98 99 100 101 102 103 104 105 106 107 108 > 109 110 111 112 113 114 115 116 117 118 119 120 > 121 122 123 124 125 126 127 > node 8 size: 261752 MB > node 8 free: 240933 MB > node distances: > node 0 8 > 0: 10 40 > 8: 40 10 > > # numastat -p $pid > Per-node process memory usage (in MBs) for PID 34168 (support_numa) > Node 0 Node 8 Total > --------------- --------------- --------------- > Huge 0.00 0.00 0.00 > Heap 0.00 0.12 0.12 > Stack 0.00 0.06 0.06 > Private 1.62 1.50 3.12 > ---------------- --------------- --------------- --------------- > Total 1.62 1.69 3.31 > > Signed-off-by: Li Wang <liwang@redhat.com> > Cc: Chunyu Hu <chuhu@redhat.com> > Cc: Cyril Hrubis <chrubis@suse.cz> > --- > testcases/kernel/numa/numa01.sh | 26 ++++++++++++++++++++++---- > 1 file changed, 22 insertions(+), 4 deletions(-) > > diff --git a/testcases/kernel/numa/numa01.sh b/testcases/kernel/numa/numa01.sh > index 33393ac8d..7f3dee849 100755 > --- a/testcases/kernel/numa/numa01.sh > +++ b/testcases/kernel/numa/numa01.sh > @@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat" > extract_numastat_p() > { > local pid=$1 > - local node=$(($2 + 2)) > - > - echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}') > + local node=$2 > + > + echo $(numastat -p $pid | \ > + awk -v node=$node '/Node/ { \ > + gsub("Node", ""); \ > + for (i=0; i<NF; i++) \ > + if ($i == node) \ > + col=i+1; \ > + next \ > + } \ > + /^Total/ {print $col}' \ > + ) > } If we can use extract_numastat_p0 with node_index then the existing code works, for example if can index while iterating $node_list and use this with extract_numastat_p0 then it should work. This is just my thoughts, please feel free to correct me if I miss or it is something wrong. Thanks! -- Bala > > check_for_support_numa() > @@ -363,7 +372,16 @@ test9() > pid=$! > TST_RETRY_FUNC "check_for_support_numa $pid" 0 > > - Mem_huge=$(echo $(numastat -p $pid |awk '/^Huge/ {print $'$((node+2))'}')) > + Mem_huge=$(echo $(numastat -p $pid | \ > + awk -v node=$node '/Node/ { \ > + gsub("Node", ""); \ > + for (i=0; i<NF; i++) \ > + if ($i == node) \ > + col=i+1; \ > + next \ > + } \ > + /^Huge/ {print $col}') \ > + ) > Mem_huge=$((${Mem_huge%.*} * 1024)) > > if [ "$Mem_huge" -lt "$HPAGE_SIZE" ]; then > -- > 2.20.1 > > > -- > Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes 2019-05-09 7:16 ` [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes Balamuruhan S @ 2019-05-09 7:54 ` Li Wang 2019-05-09 9:19 ` Balamuruhan S 0 siblings, 1 reply; 13+ messages in thread From: Li Wang @ 2019-05-09 7:54 UTC (permalink / raw) To: ltp On Thu, May 9, 2019 at 3:16 PM Balamuruhan S <bala24@linux.vnet.ibm.com> wrote: > On Wed, May 08, 2019 at 04:44:46PM +0800, Li Wang wrote: > > Numa test failed on such machine which has non-continuous numa nodes, > > it gets wrong data because of the below syntax rule is not applicable > > to that special situation. > > ` numastat -p $pid |awk '/^Total/ {print $'$((node+2))'}' > > In this patch, we fix that via a bit complex way of awk to get the > > allocated memory in specified node. > > > > # numactl -H > > available: 2 nodes (0,8) > > node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 > > 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 > > 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 > > 51 52 53 54 55 56 57 58 59 60 61 62 63 > > node 0 size: 257741 MB > > node 0 free: 253158 MB > > node 8 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 > > 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 > > 96 97 98 99 100 101 102 103 104 105 106 107 108 > > 109 110 111 112 113 114 115 116 117 118 119 120 > > 121 122 123 124 125 126 127 > > node 8 size: 261752 MB > > node 8 free: 240933 MB > > node distances: > > node 0 8 > > 0: 10 40 > > 8: 40 10 > > > > # numastat -p $pid > > Per-node process memory usage (in MBs) for PID 34168 (support_numa) > > Node 0 Node 8 Total > > --------------- --------------- --------------- > > Huge 0.00 0.00 0.00 > > Heap 0.00 0.12 0.12 > > Stack 0.00 0.06 0.06 > > Private 1.62 1.50 3.12 > > ---------------- --------------- --------------- --------------- > > Total 1.62 1.69 3.31 > > > > Signed-off-by: Li Wang <liwang@redhat.com> > > Cc: Chunyu Hu <chuhu@redhat.com> > > Cc: Cyril Hrubis <chrubis@suse.cz> > > --- > > testcases/kernel/numa/numa01.sh | 26 ++++++++++++++++++++++---- > > 1 file changed, 22 insertions(+), 4 deletions(-) > > > > diff --git a/testcases/kernel/numa/numa01.sh > b/testcases/kernel/numa/numa01.sh > > index 33393ac8d..7f3dee849 100755 > > --- a/testcases/kernel/numa/numa01.sh > > +++ b/testcases/kernel/numa/numa01.sh > > @@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat" > > extract_numastat_p() > > { > > local pid=$1 > > - local node=$(($2 + 2)) > > - > > - echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}') > > + local node=$2 > > + > > + echo $(numastat -p $pid | \ > > + awk -v node=$node '/Node/ { \ > > + gsub("Node", ""); \ > > + for (i=0; i<NF; i++) \ > > + if ($i == node) \ > > + col=i+1; \ > > + next \ > > + } \ > > + /^Total/ {print $col}' \ > > + ) > > } > > If we can use extract_numastat_p0 with node_index then the existing code > works, for example if can index while iterating $node_list and use this > with extract_numastat_p0 then it should work. > Hi Bala, Sorry, I don't fully understand what's you mean here :(. Could you explain something more for this method? > > This is just my thoughts, please feel free to correct me if I miss or > it is something wrong. Thanks! > > -- Bala > > > > check_for_support_numa() > > @@ -363,7 +372,16 @@ test9() > > pid=$! > > TST_RETRY_FUNC "check_for_support_numa $pid" 0 > > > > - Mem_huge=$(echo $(numastat -p $pid |awk '/^Huge/ {print > $'$((node+2))'}')) > > + Mem_huge=$(echo $(numastat -p $pid | \ > > + awk -v node=$node '/Node/ { \ > > + gsub("Node", ""); \ > > + for (i=0; i<NF; i++) \ > > + if ($i == node) \ > > + col=i+1; \ > > + next \ > > + } \ > > + /^Huge/ {print $col}') \ > > + ) > > Mem_huge=$((${Mem_huge%.*} * 1024)) > > > > if [ "$Mem_huge" -lt "$HPAGE_SIZE" ]; then > > -- > > 2.20.1 > > > > > > -- > > Mailing list info: https://lists.linux.it/listinfo/ltp > > -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20190509/81d3c6c9/attachment.html> ^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes 2019-05-09 7:54 ` Li Wang @ 2019-05-09 9:19 ` Balamuruhan S 2019-05-09 10:00 ` Li Wang 0 siblings, 1 reply; 13+ messages in thread From: Balamuruhan S @ 2019-05-09 9:19 UTC (permalink / raw) To: ltp On Thu, May 09, 2019 at 03:54:15PM +0800, Li Wang wrote: > On Thu, May 9, 2019 at 3:16 PM Balamuruhan S <bala24@linux.vnet.ibm.com> > wrote: > > > On Wed, May 08, 2019 at 04:44:46PM +0800, Li Wang wrote: > > > Numa test failed on such machine which has non-continuous numa nodes, > > > it gets wrong data because of the below syntax rule is not applicable > > > to that special situation. > > > ` numastat -p $pid |awk '/^Total/ {print $'$((node+2))'}' > > > In this patch, we fix that via a bit complex way of awk to get the > > > allocated memory in specified node. > > > > > > # numactl -H > > > available: 2 nodes (0,8) > > > node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 > > > 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 > > > 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 > > > 51 52 53 54 55 56 57 58 59 60 61 62 63 > > > node 0 size: 257741 MB > > > node 0 free: 253158 MB > > > node 8 cpus: 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 > > > 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 > > > 96 97 98 99 100 101 102 103 104 105 106 107 108 > > > 109 110 111 112 113 114 115 116 117 118 119 120 > > > 121 122 123 124 125 126 127 > > > node 8 size: 261752 MB > > > node 8 free: 240933 MB > > > node distances: > > > node 0 8 > > > 0: 10 40 > > > 8: 40 10 > > > > > > # numastat -p $pid > > > Per-node process memory usage (in MBs) for PID 34168 (support_numa) > > > Node 0 Node 8 Total > > > --------------- --------------- --------------- > > > Huge 0.00 0.00 0.00 > > > Heap 0.00 0.12 0.12 > > > Stack 0.00 0.06 0.06 > > > Private 1.62 1.50 3.12 > > > ---------------- --------------- --------------- --------------- > > > Total 1.62 1.69 3.31 > > > > > > Signed-off-by: Li Wang <liwang@redhat.com> > > > Cc: Chunyu Hu <chuhu@redhat.com> > > > Cc: Cyril Hrubis <chrubis@suse.cz> > > > --- > > > testcases/kernel/numa/numa01.sh | 26 ++++++++++++++++++++++---- > > > 1 file changed, 22 insertions(+), 4 deletions(-) > > > > > > diff --git a/testcases/kernel/numa/numa01.sh > > b/testcases/kernel/numa/numa01.sh > > > index 33393ac8d..7f3dee849 100755 > > > --- a/testcases/kernel/numa/numa01.sh > > > +++ b/testcases/kernel/numa/numa01.sh > > > @@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat" > > > extract_numastat_p() > > > { > > > local pid=$1 > > > - local node=$(($2 + 2)) > > > - > > > - echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}') > > > + local node=$2 > > > + > > > + echo $(numastat -p $pid | \ > > > + awk -v node=$node '/Node/ { \ > > > + gsub("Node", ""); \ > > > + for (i=0; i<NF; i++) \ > > > + if ($i == node) \ > > > + col=i+1; \ > > > + next \ > > > + } \ > > > + /^Total/ {print $col}' \ > > > + ) > > > } > > > > If we can use extract_numastat_p0 with node_index then the existing code > > works, for example if can index while iterating $node_list and use this > > with extract_numastat_p0 then it should work. > > > > Hi Bala, > > Sorry, I don't fully understand what's you mean here :(. Could you explain > something more for this method? instead of changing `extract_numastat_p0()` can do something like this ? diff --git a/testcases/kernel/numa/numa01.sh b/testcases/kernel/numa/numa01.sh index 33393ac8d..47c18edd6 100755 --- a/testcases/kernel/numa/numa01.sh +++ b/testcases/kernel/numa/numa01.sh @@ -94,6 +94,7 @@ setup() test1() { Mem_curr=0 + node_index=0 for node in $nodes_list; do numactl --cpunodebind=$node --membind=$node support_numa alloc_1MB & @@ -101,7 +102,8 @@ test1() TST_RETRY_FUNC "check_for_support_numa $pid" 0 - Mem_curr=$(echo "$(extract_numastat_p $pid $node) * $MB" |bc) + Mem_curr=$(echo "$(extract_numastat_p $pid $node_index) * $MB" |bc) + let node_index++ if [ $(echo "$Mem_curr < $MB" | bc) -eq 1 ]; then tst_res TFAIL \ "NUMA memory allocated in node$node is less than expected" -- Bala > > > > > > > This is just my thoughts, please feel free to correct me if I miss or > > it is something wrong. Thanks! > > > > -- Bala > > > > > > check_for_support_numa() > > > @@ -363,7 +372,16 @@ test9() > > > pid=$! > > > TST_RETRY_FUNC "check_for_support_numa $pid" 0 > > > > > > - Mem_huge=$(echo $(numastat -p $pid |awk '/^Huge/ {print > > $'$((node+2))'}')) > > > + Mem_huge=$(echo $(numastat -p $pid | \ > > > + awk -v node=$node '/Node/ { \ > > > + gsub("Node", ""); \ > > > + for (i=0; i<NF; i++) \ > > > + if ($i == node) \ > > > + col=i+1; \ > > > + next \ > > > + } \ > > > + /^Huge/ {print $col}') \ > > > + ) > > > Mem_huge=$((${Mem_huge%.*} * 1024)) > > > > > > if [ "$Mem_huge" -lt "$HPAGE_SIZE" ]; then > > > -- > > > 2.20.1 > > > > > > > > > -- > > > Mailing list info: https://lists.linux.it/listinfo/ltp > > > > > > -- > Regards, > Li Wang ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes 2019-05-09 9:19 ` Balamuruhan S @ 2019-05-09 10:00 ` Li Wang 2019-05-14 15:06 ` Cyril Hrubis 0 siblings, 1 reply; 13+ messages in thread From: Li Wang @ 2019-05-09 10:00 UTC (permalink / raw) To: ltp Hi Bala, On Thu, May 9, 2019 at 5:19 PM Balamuruhan S <bala24@linux.vnet.ibm.com> wrote: > [...snip...] > > > > --- a/testcases/kernel/numa/numa01.sh > > > > +++ b/testcases/kernel/numa/numa01.sh > > > > @@ -52,9 +52,18 @@ TST_NEEDS_CMDS="awk bc numactl numastat" > > > > extract_numastat_p() > > > > { > > > > local pid=$1 > > > > - local node=$(($2 + 2)) > > > > - > > > > - echo $(numastat -p $pid |awk '/^Total/ {print $'$node'}') > > > > + local node=$2 > > > > + > > > > + echo $(numastat -p $pid | \ > > > > + awk -v node=$node '/Node/ { \ > > > > + gsub("Node", ""); \ > > > > + for (i=0; i<NF; i++) \ > > > > + if ($i == node) \ > > > > + col=i+1; \ > > > > + next \ > > > > + } \ > > > > + /^Total/ {print $col}' \ > > > > + ) > > > > } > > > > > > If we can use extract_numastat_p0 with node_index then the existing > code > > > works, for example if can index while iterating $node_list and use this > > > with extract_numastat_p0 then it should work. > > > > > > > Hi Bala, > > > > Sorry, I don't fully understand what's you mean here :(. Could you > explain > > something more for this method? > > instead of changing `extract_numastat_p0()` can do something like this ? > diff --git a/testcases/kernel/numa/numa01.sh > b/testcases/kernel/numa/numa01.sh > index 33393ac8d..47c18edd6 100755 > --- a/testcases/kernel/numa/numa01.sh > +++ b/testcases/kernel/numa/numa01.sh > @@ -94,6 +94,7 @@ setup() > test1() > { > Mem_curr=0 > + node_index=0 > > for node in $nodes_list; do > numactl --cpunodebind=$node --membind=$node support_numa > alloc_1MB & > @@ -101,7 +102,8 @@ test1() > > TST_RETRY_FUNC "check_for_support_numa $pid" 0 > > - Mem_curr=$(echo "$(extract_numastat_p $pid $node) * $MB" > |bc) > + Mem_curr=$(echo "$(extract_numastat_p $pid $node_index) * > $MB" |bc) > + let node_index++ > I guess it can be work, but the disadvantage of that is we have to involve a new variable(node_index) in each of the tests (from test1 to test10). Hence I don't think it is much better than my patch. For which way to go, I'd leave this to Cyril to make a choice. Or, maybe he has different thoughts on this:). Anyway, thanks for your suggestion. -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20190509/55f8a078/attachment.html> ^ permalink raw reply [flat|nested] 13+ messages in thread
* [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes 2019-05-09 10:00 ` Li Wang @ 2019-05-14 15:06 ` Cyril Hrubis 2021-11-10 10:54 ` Richard Palethorpe 0 siblings, 1 reply; 13+ messages in thread From: Cyril Hrubis @ 2019-05-14 15:06 UTC (permalink / raw) To: ltp Hi! > > diff --git a/testcases/kernel/numa/numa01.sh > > b/testcases/kernel/numa/numa01.sh > > index 33393ac8d..47c18edd6 100755 > > --- a/testcases/kernel/numa/numa01.sh > > +++ b/testcases/kernel/numa/numa01.sh > > @@ -94,6 +94,7 @@ setup() > > test1() > > { > > Mem_curr=0 > > + node_index=0 > > > > for node in $nodes_list; do > > numactl --cpunodebind=$node --membind=$node support_numa > > alloc_1MB & > > @@ -101,7 +102,8 @@ test1() > > > > TST_RETRY_FUNC "check_for_support_numa $pid" 0 > > > > - Mem_curr=$(echo "$(extract_numastat_p $pid $node) * $MB" > > |bc) > > + Mem_curr=$(echo "$(extract_numastat_p $pid $node_index) * > > $MB" |bc) > > + let node_index++ > > > > I guess it can be work, but the disadvantage of that is we have to involve > a new variable(node_index) in each of the tests (from test1 to test10). > Hence I don't think it is much better than my patch. For which way to go, > I'd leave this to Cyril to make a choice. Or, maybe he has different > thoughts on this:). I actually do not care that much about the numa01.sh tests, because these are broken in more ways than this and were never correct to begin with. I've started to rewrite these into proper tests, the set_mempolicy() was first part of that effort, the mbind() tests are continuation of that and the end goal is to get rid of these broken tests eventually. -- Cyril Hrubis chrubis@suse.cz ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes 2019-05-14 15:06 ` Cyril Hrubis @ 2021-11-10 10:54 ` Richard Palethorpe 0 siblings, 0 replies; 13+ messages in thread From: Richard Palethorpe @ 2021-11-10 10:54 UTC (permalink / raw) To: Cyril Hrubis; +Cc: ltp Hello, Cyril Hrubis <chrubis@suse.cz> writes: > Hi! >> > diff --git a/testcases/kernel/numa/numa01.sh >> > b/testcases/kernel/numa/numa01.sh >> > index 33393ac8d..47c18edd6 100755 >> > --- a/testcases/kernel/numa/numa01.sh >> > +++ b/testcases/kernel/numa/numa01.sh >> > @@ -94,6 +94,7 @@ setup() >> > test1() >> > { >> > Mem_curr=0 >> > + node_index=0 >> > >> > for node in $nodes_list; do >> > numactl --cpunodebind=$node --membind=$node support_numa >> > alloc_1MB & >> > @@ -101,7 +102,8 @@ test1() >> > >> > TST_RETRY_FUNC "check_for_support_numa $pid" 0 >> > >> > - Mem_curr=$(echo "$(extract_numastat_p $pid $node) * $MB" >> > |bc) >> > + Mem_curr=$(echo "$(extract_numastat_p $pid $node_index) * >> > $MB" |bc) >> > + let node_index++ >> > >> >> I guess it can be work, but the disadvantage of that is we have to involve >> a new variable(node_index) in each of the tests (from test1 to test10). >> Hence I don't think it is much better than my patch. For which way to go, >> I'd leave this to Cyril to make a choice. Or, maybe he has different >> thoughts on this:). > > I actually do not care that much about the numa01.sh tests, because > these are broken in more ways than this and were never correct to begin > with. > > I've started to rewrite these into proper tests, the set_mempolicy() was > first part of that effort, the mbind() tests are continuation of that > and the end goal is to get rid of these broken tests eventually. We have quite a few mbind and set_mempolicy tests now. So should we delete this test? > > -- > Cyril Hrubis > chrubis@suse.cz -- Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2021-11-10 10:57 UTC | newest] Thread overview: 13+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-05-08 8:44 [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes Li Wang 2019-05-08 8:44 ` [LTP] [PATCH v2 2/2] libltpnuma: remove restrictions on numa node-id Li Wang 2019-05-09 6:45 ` Balamuruhan S 2019-05-09 8:00 ` Li Wang 2019-05-09 9:23 ` Balamuruhan S 2019-05-09 15:50 ` Jan Stancek 2019-05-14 14:30 ` Cyril Hrubis 2019-05-09 7:16 ` [LTP] [PATCH v2 1/2] numa: fix numa test error with non-continuous nodes Balamuruhan S 2019-05-09 7:54 ` Li Wang 2019-05-09 9:19 ` Balamuruhan S 2019-05-09 10:00 ` Li Wang 2019-05-14 15:06 ` Cyril Hrubis 2021-11-10 10:54 ` Richard Palethorpe
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox