Re: [PATCH 0/2] non-contiguous NUMA nodes handling in numactl and libnuma

[Cliff Wickman <cpw@sgi.com>]

Hi Amit,

Thanks much for the patches.

I tested them this morning.  The standard tests pass on ia64 (8 nodes).

The patches are included in the numactl-2.0.3-rc2.tar.gz tarball at
ftp://oss.sgi.com/www/projects/libnuma/download/

Any other review and testing is welcome.

-Cliff

On Mon, Mar 02, 2009 at 06:35:46PM +0530, Amit K. Arora wrote:
> PROBLEM: If the NUMA nodes in a system are not contiguous, numactl shows
> incorrect output. Here is an example from one such system:
> 
> ------------------------------------------------------
> # ls /sys/devices/system/node
> has_cpu  has_normal_memory  node0  node1  node4  node5  online  possible
> # numactl --hardware
> available: 6 nodes (0-5)
> node 0 cpus: 0 1 2 3
> node 0 size: 15232 MB
> node 0 free: 2265 MB
> node 1 cpus: 4 5 6 7
> node 1 size: 16256 MB
> node 1 free: 3108 MB
> libnuma: Warning: /sys not mounted or invalid. Assuming one node: No
> such file or directory
> node 2 cpus:
> node 2 size: <not available>
> node 2 free: <not available>
> node 3 cpus:
> node 3 size: <not available>
> node 3 free: <not available>
> node 4 cpus: 8 9 10 11
> node 4 size: 16000 MB
> node 4 free: 9292 MB
> node 5 cpus: 12 13 14 15
> node 5 size: 16128 MB
> node 5 free: 9479 MB
> node distances:
> node   0   1   2   3   4   5 
>   0:  10  20  20  20  20  10 
>   1:  20  10  20  20   0   0 
>   2:   0   0   0   0   0   0 
>   3:   0   0   0   0   0   0 
>   4:   0   0   0  1112145  1024  2607344 
>   5:  1024  2607344  1024  2607344   0   0 
> ------------------------------------------------------
> 
> 
> As we see above, there are three problems with above output:
> o it is showing wrong number of available nodes
> 	Actually there are 4 nodes available, but it says "6".
> o it tries to access information for non-exsistent nodes from /sys
> 	Thus displaying a wrong warning message too
> o the NUMA distances it shows above are clearly incorrect.
> 
> Following two patches fixes these problems:
> 1) numactl-handle-NUMA-dist-properly.patch
> 2) numactl-ignore-nonavailable-nodes.patch
> 
> 
> --
> Regards,
> Amit Arora
> --
> To unsubscribe from this list: send the line "unsubscribe linux-numa" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Cliff Wickman
Silicon Graphics, Inc.
cpw@sgi.com
(651) 683-3824