Hi Cliff, and others, I'm seeking support for us to rework numastat over the next few months to add functionality to show more per NUMA node statistics for applications and the general system. See the attached program for examples of the functionality I hope to add. Andi seems OK with it, so long as we make the default behavior the same as today's script, which we will certainly try to do. What do people think? Please advise. Thanks! - Bill -------- Original Message -------- Subject: RE: numastat tool Date: Mon, 4 Jun 2012 20:45:05 +0000 From: Kleen, Andi To: Bill Gray CC: Douglas Shakshober , Bill Burns >Assuming we make the default behavior the same, etc., what would you >think about us "enhancing" numastat to also include the functionality of >the attached code? Sounds good. It's just a perl script today, but turning it into C should be fine. And better information is always good. Just default should stay to same to not break scripts. I don't maintain numactl anymore though, so you would need to ask Cliff Wickman / linux-numa@vger.kernel.org -Andi -------- Original Message -------- Subject: numastat tool Date: Mon, 04 Jun 2012 16:36:23 -0400 From: Bill Gray To: Kleen, Andi CC: Douglas Shakshober , Bill Burns Hi Andi, We're working on a tool to show per-node usage of memory for applications and the system as a whole. We're hoping to get the tool in RHEL 6.4. See the attached source code -- try it out if you like. Basically, it adds up /proc info and displays it on a per-node basis. The tool is currently called "nmstat", but "numastat" is such a better name.... ;-) Assuming we make the default behavior the same, etc., what would you think about us "enhancing" numastat to also include the functionality of the attached code? Please let us know your thoughts. Thank you! - Bill ---------- following output demonstrates nmstat showing effectiveness of numad moving some "pig" test processes.... Hopefully email won't horribly fold the lines ------------ # ./nmstat -V ./nmstat version: 20120412: Jun 4 2012 # ./nmstat 1 Per-node process memory usage (in MBs) for PID 1 (init): Node 0 Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Node 7 ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- huge 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 heap 0.02 0.00 0.01 0.00 0.10 0.00 0.01 0.02 stack 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 private 0.70 0.00 0.00 0.01 0.63 0.00 0.00 0.02 total 0.72 0.00 0.01 0.01 0.74 0.00 0.01 0.04 # ./nmstat -n pig Per-node process memory usage (in MBs): PID Node 0 Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Node 7 ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- 22075 (pig) 0.08 2998.34 0.00 0.00 0.10 0.00 1002.20 1000.00 22076 (pig) 1000.04 0.11 2001.92 0.00 998.19 0.00 1000.13 0.00 22077 (pig) 0.04 1001.96 0.00 2000.07 998.19 0.00 0.13 1000.00 22078 (pig) 1000.04 0.11 2003.71 998.15 0.09 1000.00 0.13 0.00 22079 (pig) 1001.89 0.14 0.00 0.00 1000.04 2001.89 0.13 998.15 # numad -S 0 -p 22075 -p 22076 -p 22077 { this tool moves processes around for NUMA affinity } ... a minute passes .... # numad -i0 { turn off numad } # ./nmstat -n pig Per-node process memory usage (in MBs): PID Node 0 Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Node 7 ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- 22075 (pig) 0.00 5000.43 0.00 0.00 0.00 0.00 0.29 0.00 22076 (pig) 0.00 0.00 0.00 0.00 0.00 0.00 5000.39 0.00 22077 (pig) 0.00 0.00 0.00 5000.10 0.00 0.00 0.29 0.00 22078 (pig) 1000.00 0.03 2003.71 998.15 0.06 1000.00 0.29 0.00 22079 (pig) 1001.86 0.05 0.00 0.00 1000.00 2001.89 0.29 998.15