Re: [PATCH 1/2] x86/cpu hotplug: make apicid <--> cpuid mapping persistent

[Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>]

On 2015/03/30 18:58, Gu Zheng wrote:
> Hi Kame-san,
> 
> On 03/27/2015 12:31 AM, Kamezawa Hiroyuki wrote:
> 
>> On 2015/03/26 13:55, Gu Zheng wrote:
>>> Hi Kame-san,
>>> On 03/26/2015 11:19 AM, Kamezawa Hiroyuki wrote:
>>>
>>>> On 2015/03/26 11:17, Gu Zheng wrote:
>>>>> Previously, we build the apicid <--> cpuid mapping when the cpu is present, but
>>>>> the relationship will be changed if the cpu/node hotplug happenned, because we
>>>>> always choose the first free cpuid for the hot added cpu (whether it is new-add
>>>>> or re-add), so this the cpuid <--> node mapping changed if node hot plug
>>>>> occurred, and it causes the wq sub-system allocation failture:
>>>>>      ==
>>>>>         SLUB: Unable to allocate memory on node 2 (gfp=0x80d0)
>>>>>          cache: kmalloc-192, object size: 192, buffer size: 192, default
>>>>> order:
>>>>>        1, min order: 0
>>>>>          node 0: slabs: 6172, objs: 259224, free: 245741
>>>>>          node 1: slabs: 3261, objs: 136962, free: 127656
>>>>>      ==
>>>>> So here we build the persistent [lapic id] <--> cpuid mapping when the cpu first
>>>>> present, and never change it.
>>>>>
>>>>> Suggested-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>>>>> Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
>>>>> ---
>>>>>     arch/x86/kernel/apic/apic.c |   31 ++++++++++++++++++++++++++++++-
>>>>>     1 files changed, 30 insertions(+), 1 deletions(-)
>>>>>
>>>>> diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
>>>>> index ad3639a..d539ebc 100644
>>>>> --- a/arch/x86/kernel/apic/apic.c
>>>>> +++ b/arch/x86/kernel/apic/apic.c
>>>>> @@ -2038,6 +2038,30 @@ void disconnect_bsp_APIC(int virt_wire_setup)
>>>>>     	apic_write(APIC_LVT1, value);
>>>>>     }
>>>>>     
>>>>> +/*
>>>>> + * Logic cpu number(cpuid) to local APIC id persistent mappings.
>>>>> + * Do not clear the mapping even if cpu hot removed.
>>>>> + * */
>>>>> +static int apicid_to_x86_cpu[MAX_LOCAL_APIC] = {
>>>>> +	[0 ... MAX_LOCAL_APIC - 1] = -1,
>>>>> +};
>>>>
>>>>
>>>> This patch cannot handle x2apic, which is 32bit.
>>>
>>> IMO, if the apicid is too big (larger than MAX_LOCAL_APIC), we will skip
>>> generating a logic cpu number for it, so it seems no problem here.
>>>
>> you mean MAX_LOCAL_APIC=32768 ? ....isn't it too wasting ?
> 
> I use the big array here to keep the same format with the existed ones:
> int apic_version[MAX_LOCAL_APIC];
> 
> s16 __apicid_to_node[MAX_LOCAL_APIC] = {
> 	[0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE
> };
> Or we should also say "NO" to them?

This will not survive when someone try to enlarge MAX_LOCAL_APIC for
introdcing new cpu model because x2apic is 32bit by definition.

Please create long surviving infrastructure.

Thanks,
-Kame