Re: [RFC Patch V1 28/30] mm: Update _mem_id_[] for every possible CPU when memory configuration changes

[Nishanth Aravamudan <nacc@linux.vnet.ibm.com>]

On 11.07.2014 [15:37:45 +0800], Jiang Liu wrote:
> Current kernel only updates _mem_id_[cpu] for onlined CPUs when memory
> configuration changes. So kernel may allocate memory from remote node
> for a CPU if the CPU is still in absent or offline state even if the
> node associated with the CPU has already been onlined.

This just sounds like the topology information is being updated at the
wrong place/time? That is, the memory is online, the CPU is being
brought online, but isn't associated with any node?

> This patch tries to improve performance by updating _mem_id_[cpu] for
> each possible CPU when memory configuration changes, thus kernel could
> always allocate from local node once the node is onlined.

Ok, what is the impact? Do you actually see better performance?

> We check node_online(cpu_to_node(cpu)) because:
> 1) local_memory_node(nid) needs to access NODE_DATA(nid)
> 2) try_offline_node(nid) just zeroes out NODE_DATA(nid) instead of free it
> 
> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
> ---
>  mm/page_alloc.c |   10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 0ea758b898fd..de86e941ed57 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3844,13 +3844,13 @@ static int __build_all_zonelists(void *data)
>  		/*
>  		 * We now know the "local memory node" for each node--
>  		 * i.e., the node of the first zone in the generic zonelist.
> -		 * Set up numa_mem percpu variable for on-line cpus.  During
> -		 * boot, only the boot cpu should be on-line;  we'll init the
> -		 * secondary cpus' numa_mem as they come on-line.  During
> -		 * node/memory hotplug, we'll fixup all on-line cpus.
> +		 * Set up numa_mem percpu variable for all possible cpus
> +		 * if associated node has been onlined.
>  		 */
> -		if (cpu_online(cpu))
> +		if (node_online(cpu_to_node(cpu)))
>  			set_cpu_numa_mem(cpu, local_memory_node(cpu_to_node(cpu)));
> +		else
> +			set_cpu_numa_mem(cpu, NUMA_NO_NODE);
>  #endif