Re: [2.6.24-rc8-mm1][regression?] numactl --interleave=all doesn't works on memoryless node.

Andi Kleen
2008-02-02T08:34:52Z

[intentional full quote]

On Sat, Feb 02, 2008 at 05:12:30PM +0900, KOSAKI Motohiro wrote:
> I tested numactl on 2.6.24-rc8-mm1.
> and I found strange behavior.
>
> test method and result.
>
> $ numactl --interleave=all ls
> set_mempolicy: Invalid argument
> setting interleave mask: Invalid argument
>
> numactl command download from
> ftp://ftp.suse.com/pub/people/ak/numa/
> (I choice numactl-1.0.2)
>
>
> Of course, older kernel(RHEL5.1) works good.
>
>
>
> more detail:
>
> 1. my machine node and memory.
>
> $ numactl --hardware
> available: 16 nodes (0-15)
> node 0 size: 0 MB
> node 0 free: 0 MB
> node 1 size: 0 MB
> node 1 free: 0 MB
> node 2 size: 3872 MB
> node 2 free: 1487 MB
> node 3 size: 4032 MB
> node 3 free: 3671 MB
> node 4 size: 0 MB
> node 4 free: 0 MB
> node 5 size: 0 MB
> node 5 free: 0 MB
> node 6 size: 0 MB
> node 6 free: 0 MB
> node 7 size: 0 MB
> node 7 free: 0 MB
> node 8 size: 0 MB
> node 8 free: 0 MB
> node 9 size: 0 MB
> node 9 free: 0 MB
> node 10 size: 0 MB
> node 10 free: 0 MB
> node 11 size: 0 MB
> node 11 free: 0 MB
> node 12 size: 0 MB
> node 12 free: 0 MB
> node 13 size: 0 MB
> node 13 free: 0 MB
> node 14 size: 0 MB
> node 14 free: 0 MB
> node 15 size: 0 MB
> node 15 free: 0 MB
>
>
> 2. numactl behavior of --interleave=all
> 2.1 scan "/sys/devices/system/node" dir
> 2.2 calculate max node number
> 2.3 all bit turn on of existing node.
> (i.e. 0xFF generated on my environment.)
> 2.4 call set_mempolicy()
>
> 3. 2.6.24-rc8-mm1 set_mempolicy(2) behavior
> 3.1 check nodesubset(nodemask argument, node_states[N_HIGH_MEMORY])
> in mpol_check_policy()
>
> -> check failed when memmoryless node exist.
> (i.e. node_states[N_HIGH_MEMORY] of my machine is 0xc)
>
> 4. RHEL5.1 set_mempolicy(2) behavior
> 4.1 check nodesubset(nodemask argument, node_online_map)
> in mpol_check_policy().
>
> -> check success.
>
>
> I don't know wrong either kernel or libnuma.

When the kernel behaviour changes and breaks user space then the kernel
is usually wrong. Cc'ed Lee S. who maintains the kernel code now.

-Andi