From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753411AbbDAC7t (ORCPT ); Tue, 31 Mar 2015 22:59:49 -0400 Received: from mgwkm03.jp.fujitsu.com ([202.219.69.170]:42454 "EHLO mgwkm03.jp.fujitsu.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753139AbbDAC7r (ORCPT ); Tue, 31 Mar 2015 22:59:47 -0400 X-SecurityPolicyCheck: OK by SHieldMailChecker v2.2.3 X-SHieldMailCheckerPolicyVersion: FJ-ISEC-20140219-2 Message-ID: <551B5F17.4070100@jp.fujitsu.com> Date: Wed, 1 Apr 2015 11:59:35 +0900 From: Kamezawa Hiroyuki User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Gu Zheng CC: , , , , , Subject: Re: [PATCH 1/2] x86/cpu hotplug: make apicid <--> cpuid mapping persistent References: <1427336275-32066-1-git-send-email-guz.fnst@cn.fujitsu.com> <1427336275-32066-2-git-send-email-guz.fnst@cn.fujitsu.com> <55137ADF.7070703@jp.fujitsu.com> <55139128.2050603@cn.fujitsu.com> <5514345B.40306@jp.fujitsu.com> <55191E54.3000205@cn.fujitsu.com> In-Reply-To: <55191E54.3000205@cn.fujitsu.com> Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit X-SecurityPolicyCheck-GC: OK by FENCE-Mail X-TM-AS-MML: disable Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2015/03/30 18:58, Gu Zheng wrote: > Hi Kame-san, > > On 03/27/2015 12:31 AM, Kamezawa Hiroyuki wrote: > >> On 2015/03/26 13:55, Gu Zheng wrote: >>> Hi Kame-san, >>> On 03/26/2015 11:19 AM, Kamezawa Hiroyuki wrote: >>> >>>> On 2015/03/26 11:17, Gu Zheng wrote: >>>>> Previously, we build the apicid <--> cpuid mapping when the cpu is present, but >>>>> the relationship will be changed if the cpu/node hotplug happenned, because we >>>>> always choose the first free cpuid for the hot added cpu (whether it is new-add >>>>> or re-add), so this the cpuid <--> node mapping changed if node hot plug >>>>> occurred, and it causes the wq sub-system allocation failture: >>>>> == >>>>> SLUB: Unable to allocate memory on node 2 (gfp=0x80d0) >>>>> cache: kmalloc-192, object size: 192, buffer size: 192, default >>>>> order: >>>>> 1, min order: 0 >>>>> node 0: slabs: 6172, objs: 259224, free: 245741 >>>>> node 1: slabs: 3261, objs: 136962, free: 127656 >>>>> == >>>>> So here we build the persistent [lapic id] <--> cpuid mapping when the cpu first >>>>> present, and never change it. >>>>> >>>>> Suggested-by: KAMEZAWA Hiroyuki >>>>> Signed-off-by: Gu Zheng >>>>> --- >>>>> arch/x86/kernel/apic/apic.c | 31 ++++++++++++++++++++++++++++++- >>>>> 1 files changed, 30 insertions(+), 1 deletions(-) >>>>> >>>>> diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c >>>>> index ad3639a..d539ebc 100644 >>>>> --- a/arch/x86/kernel/apic/apic.c >>>>> +++ b/arch/x86/kernel/apic/apic.c >>>>> @@ -2038,6 +2038,30 @@ void disconnect_bsp_APIC(int virt_wire_setup) >>>>> apic_write(APIC_LVT1, value); >>>>> } >>>>> >>>>> +/* >>>>> + * Logic cpu number(cpuid) to local APIC id persistent mappings. >>>>> + * Do not clear the mapping even if cpu hot removed. >>>>> + * */ >>>>> +static int apicid_to_x86_cpu[MAX_LOCAL_APIC] = { >>>>> + [0 ... MAX_LOCAL_APIC - 1] = -1, >>>>> +}; >>>> >>>> >>>> This patch cannot handle x2apic, which is 32bit. >>> >>> IMO, if the apicid is too big (larger than MAX_LOCAL_APIC), we will skip >>> generating a logic cpu number for it, so it seems no problem here. >>> >> you mean MAX_LOCAL_APIC=32768 ? ....isn't it too wasting ? > > I use the big array here to keep the same format with the existed ones: > int apic_version[MAX_LOCAL_APIC]; > > s16 __apicid_to_node[MAX_LOCAL_APIC] = { > [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE > }; > Or we should also say "NO" to them? This will not survive when someone try to enlarge MAX_LOCAL_APIC for introdcing new cpu model because x2apic is 32bit by definition. Please create long surviving infrastructure. Thanks, -Kame