Re: [RFC] powerpc/numa: Use VPHN based node ID information on shared processor LPARs

[Anshuman Khandual <khandual@linux.vnet.ibm.com>]

On 10/14/2015 02:49 PM, Michael Ellerman wrote:
> On Wed, 2015-10-14 at 14:32 +0530, Anshuman Khandual wrote:
>> On shared processor LPARs, H_HOME_NODE_ASSOCIATIVITY hcall provides the
>> dynamic virtual-physical mapping for any given processor. Currently we
>> use VPHN node ID information only after getting either a PRRN or a VPHN
>> event. But during boot time inside the function numa_setup_cpu, we still
>> query the OF device tree for the node ID value which might be different
>> than what can be fetched from the H_HOME_NODE_ASSOCIATIVITY hcall. In a
>> scenario where there are no PRRN or VPHN event after boot, all node-cpu
>> mapping will remain incorrect there after.
>>
>> With this proposed change, numa_setup_cpu will try to override the OF
>> device tree fetched node ID information with H_HOME_NODE_ASSOCIATIVITY
>> hcall fetched node ID value. Right now shared processor property of the
>> LPAR cannot be queried as VPA inializaion happens after numa_setup_cpu
>> during boot time. So initmem_init function has been moved after ppc_md.
>> setup_arch inside setup_arch during boot.
> 
> I would be *very* reluctant to change the order of initmem_init() vs
> setup_arch().
> 
> At a minimum you'd need to go through every setup_arch() implementation and
> carefully determine if the ordering of what it does matters vs initmem_init().
> And then you'd need to test on every affected platform.
> 
> So I suggest you think of a different way to do it if at all possible.

vpa_init() is being called inside pSeries_setup_arch which is ppc_md
.setup_arch for the platform. Its called directly for the boot cpu
and through smp_init_pseries_xics for other cpus on the system. Not
sure what is the reason behind calling vpa_init() from XICS init
though.

If we can move all these vpa_init() calls from pSeries_setup_arch
to initmem_init just before calling numa_setup_cpu, the VPA area
would be initialized when we need it during boot. Will look in
this direction.