Re: [PATCH] x86-32: Allocate irq stacks seperate from percpu area

[Eric Dumazet <eric.dumazet@gmail.com>]

Le mercredi 27 octobre 2010 à 17:35 +0200, Tejun Heo a écrit :

> Hmmm, okay.  Can you please print out early_cpu_to_node() output for
> each cpu from arch/x86/kernel/setup_percpu.c::setup_per_cpu_areas()?
> BTW, some clarifications.
> 
> * In the pcpu-alloc debug message, the n of [n] might not necessarily
>   match the NUMA node.
> 
> * I was confused before.  If CPU distance reported by
>   early_cpu_to_node() is greater than LOCAL_DISTANCE (ie. NUMA
>   configuration), cpus will always belong to different [n].  What gets
>   adjusted is the size of each unit.
> 
> * No matter what, here, the end result is correct.  As there's no low
>   memory on node 1, it doesn't matter how the groups are organized in
>   the first chunk as long as embedding is used.  And for other chunks,
>   pages for each cpu are allocated separatedly w/ cpu_to_node() anyway
>   so NUMA affinity will be correct, again, regardless of the group
>   organization.
> 
> Thanks.
> 

Will do in a few moment, once I recover from frozen machine  :(
(See end of this mail)

Thanks !

By the way, booting with hashdist=1 to make alloc_large_system_hash()
use vmalloc() show that only pages from node 0 were used at boot.

# grep alloc_large /proc/vmallocinfo 
0xf7a01000-0xf7a82000  528384 alloc_large_system_hash+0x144/0x1d9 pages=128 vmalloc N0=128
0xf7a83000-0xf7ac4000  266240 alloc_large_system_hash+0x144/0x1d9 pages=64 vmalloc N0=64
0xf7b11000-0xf7b32000  135168 alloc_large_system_hash+0x144/0x1d9 pages=32 vmalloc N0=32
0xf7b33000-0xf7c34000 1052672 alloc_large_system_hash+0x144/0x1d9 pages=256 vmalloc N0=256
0xf7c39000-0xf7cba000  528384 alloc_large_system_hash+0x144/0x1d9 pages=128 vmalloc N0=128
0xf7cbb000-0xf7cc0000   20480 alloc_large_system_hash+0x144/0x1d9 pages=4 vmalloc N0=4
0xf7cc1000-0xf7cc6000   20480 alloc_large_system_hash+0x144/0x1d9 pages=4 vmalloc N0=4


So I tried following experiment :

# swapoff
# numactl --membind=0 swapon -a
# grep swap /proc/vmallocinfo 
0xf9bf3000-0xf9cf4000 1052672 sys_swapon+0x4aa/0xb24 pages=256 vmalloc N0=256
# swapoff -a
# numactl --membind=1 swapon -a

<<FREEZE>>