PPC64/libnuma: getting the total number of numa nodes in the system

PPC64/libnuma: getting the total number of numa nodes in the system

[Gustavo Romero]

Hi,

On PPC64 numa node numbers are not necessarily consecutive, i.e. not all nodes
from 0 to the highest node number are available in the system. For instance:

$ numactl -H
available: 4 nodes (0-1,16-17)
node 0 cpus: 0 8 16 24 32
node 0 size: 130706 MB
node 0 free: 15825 MB
node 1 cpus: 40 48 56 64 72
node 1 size: 0 MB
node 1 free: 0 MB
node 16 cpus: 80 88 96 104 112
node 16 size: 130630 MB
node 16 free: 7473 MB
node 17 cpus: 120 128 136 144 152
node 17 size: 0 MB
node 17 free: 0 MB
node distances:
node   0   1  16  17
  0:  10  20  40  40
  1:  20  10  40  40
 16:  40  40  10  20
 17:  40  40  20  10

In libnuma it's possible to get the correct number of configured nodes (2 in
that case: nodes 0 and 16) by numa_num_configured_nodes(), by numa_get_membind()
and counting the number of nodes returned, or by counting the number of nodes in
numa_all_nodes_ptr (which is listed in libnuma's manual [1]).

However to get the total number of nodes in the system (4 in that case: node 0,
1, 16, and 17) I could not find an appropriate function in libnuma's manual.
Usually it seems that on x64 it's true that that number is simply
numa_max_node() + 1 but that does not apply for PPC64 (17 + 1 != 4). Currently,
based on libnuma.c code, I'm counting the number of nodes in numa_nodes_ptr [2]
to find out the total number of nodes in the system, but that pointer is not
listed in the manual (and also it's a little bit strange - taking into account
the names of the variables - that "numa_all_nodes_ptr" contains just the
configured nodes (nodes just with memory) and that "numa_nodes_ptr" on the other
hand contains all the nodes in the system (including nodes without memory)).

So what's the preferred way to get the total number of nodes (even numa nodes
not configured, i.e. even nodes without memory) in an application that uses the
libnuma?

Thank you.

Best regards,
Gustavo

[1] https://github.com/numactl/numactl/blob/master/numa.3#L294
[2] https://github.com/numactl/numactl/blob/master/numa.h#L155