From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kefeng Wang Date: Tue, 30 Nov 2021 06:27:24 +0000 Subject: Re: [PATCH RFC 3/4] mm: percpu: Add generic pcpu_fc_alloc/free funciton Message-Id: List-Id: References: <20211121093557.139034-1-wangkefeng.wang@huawei.com> <20211121093557.139034-4-wangkefeng.wang@huawei.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable To: Dennis Zhou Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, tj@kernel.org, gregkh@linuxfoundation.org, cl@linux.com, catalin.marinas@arm.com, will@kernel.org, tsbogend@alpha.franken.de, mpe@ellerman.id.au, benh@kernel.crashing.org, paulus@samba.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, sparclinux@vger.kernel.org, x86@kernel.org On 2021/11/30 6:45, Dennis Zhou wrote: > On Sun, Nov 21, 2021 at 05:35:56PM +0800, Kefeng Wang wrote: >> With previous patch, we could add a generic pcpu first chunk >> allocation and free function to cleanup the duplicated definations >> on each architecture. >> >> Signed-off-by: Kefeng Wang >> --- >> arch/mips/mm/init.c | 16 +-------- >> arch/powerpc/kernel/setup_64.c | 51 ++------------------------ >> arch/sparc/kernel/smp_64.c | 50 +------------------------- >> arch/x86/kernel/setup_percpu.c | 59 +----------------------------- >> drivers/base/arch_numa.c | 19 +--------- >> include/linux/percpu.h | 9 +---- >> mm/percpu.c | 66 ++++++++++++++++++---------------- >> 7 files changed, 42 insertions(+), 228 deletions(-) ... >> diff --git a/include/linux/percpu.h b/include/linux/percpu.h >> index 41bb54715b0c..d73c97ef4ff4 100644 >> --- a/include/linux/percpu.h >> +++ b/include/linux/percpu.h >> @@ -95,9 +95,6 @@ extern const char * const pcpu_fc_names[PCPU_FC_NR]; >> extern enum pcpu_fc pcpu_chosen_fc; >> =20 >> typedef int (pcpu_fc_cpu_to_node_fn_t)(int cpu); >> -typedef void * (*pcpu_fc_alloc_fn_t)(unsigned int cpu, size_t size, siz= e_t align, >> - pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn); >> -typedef void (*pcpu_fc_free_fn_t)(void *ptr, size_t size); >> typedef void (*pcpu_fc_populate_pte_fn_t)(unsigned long addr); >> typedef int (pcpu_fc_cpu_distance_fn_t)(unsigned int from, unsigned in= t to); >> =20 >> @@ -112,15 +109,11 @@ extern void __init pcpu_setup_first_chunk(const st= ruct pcpu_alloc_info *ai, >> extern int __init pcpu_embed_first_chunk(size_t reserved_size, size_t = dyn_size, >> size_t atom_size, >> pcpu_fc_cpu_distance_fn_t cpu_distance_fn, >> - pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn, >> - pcpu_fc_alloc_fn_t alloc_fn, >> - pcpu_fc_free_fn_t free_fn); >> + pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn); >> #endif >> =20 >> #ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK >> extern int __init pcpu_page_first_chunk(size_t reserved_size, >> - pcpu_fc_alloc_fn_t alloc_fn, >> - pcpu_fc_free_fn_t free_fn, >> pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn, >> pcpu_fc_populate_pte_fn_t populate_pte_fn); >> #endif >> diff --git a/mm/percpu.c b/mm/percpu.c >> index 3f6cf1ff0be2..efaa1cbaf73d 100644 >> --- a/mm/percpu.c >> +++ b/mm/percpu.c >> @@ -2992,6 +2992,30 @@ static struct pcpu_alloc_info * __init __flatten = pcpu_build_alloc_info( >> =20 >> return ai; >> } >> + >> +static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size, size_= t align, >> + pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn) >> +{ >> + const unsigned long goal =3D __pa(MAX_DMA_ADDRESS); >> + int node =3D NUMA_NO_NODE; >> + void *ptr; >> + >> + if (cpu_to_nd_fn) >> + node =3D cpu_to_nd_fn(cpu); >> + >> + if (node =3D NUMA_NO_NODE || !node_online(node) || !NODE_DATA(node)) { >> + ptr =3D memblock_alloc_from(size, align, goal); >> + } else { >> + ptr =3D memblock_alloc_try_nid(size, align, goal, >> + MEMBLOCK_ALLOC_ACCESSIBLE, node); >> + } >> + return ptr; >> +} > My preference here would be to keep this identical to the x86 > implementation where we #ifdef CONFIG_NUMA. I will add back '#ifdef CONFIG_NUMA',=C2=A0 also will add back the=20 pr_debug/pr_info part as x86 does, > Overall this makes sense. > > Thanks, > Dennis > . From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41F6AC433F5 for ; Tue, 30 Nov 2021 06:27:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230405AbhK3Gat (ORCPT ); Tue, 30 Nov 2021 01:30:49 -0500 Received: from szxga08-in.huawei.com ([45.249.212.255]:28129 "EHLO szxga08-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230443AbhK3Gar (ORCPT ); Tue, 30 Nov 2021 01:30:47 -0500 Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4J3Byk5GSHz1DJpF; Tue, 30 Nov 2021 14:24:46 +0800 (CST) Received: from dggpemm500001.china.huawei.com (7.185.36.107) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 30 Nov 2021 14:27:26 +0800 Received: from [10.174.177.243] (10.174.177.243) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.20; Tue, 30 Nov 2021 14:27:25 +0800 Message-ID: Date: Tue, 30 Nov 2021 14:27:24 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Subject: Re: [PATCH RFC 3/4] mm: percpu: Add generic pcpu_fc_alloc/free funciton Content-Language: en-US To: Dennis Zhou CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , References: <20211121093557.139034-1-wangkefeng.wang@huawei.com> <20211121093557.139034-4-wangkefeng.wang@huawei.com> From: Kefeng Wang In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.177.243] X-ClientProxiedBy: dggeme704-chm.china.huawei.com (10.1.199.100) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org On 2021/11/30 6:45, Dennis Zhou wrote: > On Sun, Nov 21, 2021 at 05:35:56PM +0800, Kefeng Wang wrote: >> With previous patch, we could add a generic pcpu first chunk >> allocation and free function to cleanup the duplicated definations >> on each architecture. >> >> Signed-off-by: Kefeng Wang >> --- >> arch/mips/mm/init.c | 16 +-------- >> arch/powerpc/kernel/setup_64.c | 51 ++------------------------ >> arch/sparc/kernel/smp_64.c | 50 +------------------------- >> arch/x86/kernel/setup_percpu.c | 59 +----------------------------- >> drivers/base/arch_numa.c | 19 +--------- >> include/linux/percpu.h | 9 +---- >> mm/percpu.c | 66 ++++++++++++++++++---------------- >> 7 files changed, 42 insertions(+), 228 deletions(-) ... >> diff --git a/include/linux/percpu.h b/include/linux/percpu.h >> index 41bb54715b0c..d73c97ef4ff4 100644 >> --- a/include/linux/percpu.h >> +++ b/include/linux/percpu.h >> @@ -95,9 +95,6 @@ extern const char * const pcpu_fc_names[PCPU_FC_NR]; >> extern enum pcpu_fc pcpu_chosen_fc; >> >> typedef int (pcpu_fc_cpu_to_node_fn_t)(int cpu); >> -typedef void * (*pcpu_fc_alloc_fn_t)(unsigned int cpu, size_t size, size_t align, >> - pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn); >> -typedef void (*pcpu_fc_free_fn_t)(void *ptr, size_t size); >> typedef void (*pcpu_fc_populate_pte_fn_t)(unsigned long addr); >> typedef int (pcpu_fc_cpu_distance_fn_t)(unsigned int from, unsigned int to); >> >> @@ -112,15 +109,11 @@ extern void __init pcpu_setup_first_chunk(const struct pcpu_alloc_info *ai, >> extern int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size, >> size_t atom_size, >> pcpu_fc_cpu_distance_fn_t cpu_distance_fn, >> - pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn, >> - pcpu_fc_alloc_fn_t alloc_fn, >> - pcpu_fc_free_fn_t free_fn); >> + pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn); >> #endif >> >> #ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK >> extern int __init pcpu_page_first_chunk(size_t reserved_size, >> - pcpu_fc_alloc_fn_t alloc_fn, >> - pcpu_fc_free_fn_t free_fn, >> pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn, >> pcpu_fc_populate_pte_fn_t populate_pte_fn); >> #endif >> diff --git a/mm/percpu.c b/mm/percpu.c >> index 3f6cf1ff0be2..efaa1cbaf73d 100644 >> --- a/mm/percpu.c >> +++ b/mm/percpu.c >> @@ -2992,6 +2992,30 @@ static struct pcpu_alloc_info * __init __flatten pcpu_build_alloc_info( >> >> return ai; >> } >> + >> +static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size, size_t align, >> + pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn) >> +{ >> + const unsigned long goal = __pa(MAX_DMA_ADDRESS); >> + int node = NUMA_NO_NODE; >> + void *ptr; >> + >> + if (cpu_to_nd_fn) >> + node = cpu_to_nd_fn(cpu); >> + >> + if (node == NUMA_NO_NODE || !node_online(node) || !NODE_DATA(node)) { >> + ptr = memblock_alloc_from(size, align, goal); >> + } else { >> + ptr = memblock_alloc_try_nid(size, align, goal, >> + MEMBLOCK_ALLOC_ACCESSIBLE, node); >> + } >> + return ptr; >> +} > My preference here would be to keep this identical to the x86 > implementation where we #ifdef CONFIG_NUMA. I will add back '#ifdef CONFIG_NUMA',  also will add back the pr_debug/pr_info part as x86 does, > Overall this makes sense. > > Thanks, > Dennis > . From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 44D8FC433EF for ; Tue, 30 Nov 2021 06:27:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:CC:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QXjvlgr4xRBsNgmlfvpGBa3tMSgWh3W4uXF+Fu8XVQI=; b=Q9KIdfyeEq1dYS lOOKDM5bFtfR7uVAWntHD5MSUsVlY5DfvCGsRBC/bXWEEAsx9Xk3KesZamKRb0UMMfxm8sJ2yBhNz QbffqRs9WMTB7sBuWFTO8dxYIJLHWhabEubuiP5to5/hneUo73vf9S4khjC4Xi8DqbLTQNgRALlM4 v0/oP0ucZFjB3cLeduEZXPmOgiNpPnsBnnhSd2dF2wLB/1t4l1aTDLAZxK14Xw1PPV8z8HiUBpbwe /3RMnDoT9eqos/wmVbwRTaFFQfygbwdrvtot/9HjSDbZG03Hbrywg9K5uippCi48mmSb6WMMe0L+t UKmQbNCBv2p4VYQTBT6w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mrwcD-003nk9-00; Tue, 30 Nov 2021 06:27:45 +0000 Received: from szxga08-in.huawei.com ([45.249.212.255]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mrwbx-003nge-IK; Tue, 30 Nov 2021 06:27:31 +0000 Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4J3Byk5GSHz1DJpF; Tue, 30 Nov 2021 14:24:46 +0800 (CST) Received: from dggpemm500001.china.huawei.com (7.185.36.107) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 30 Nov 2021 14:27:26 +0800 Received: from [10.174.177.243] (10.174.177.243) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.20; Tue, 30 Nov 2021 14:27:25 +0800 Message-ID: Date: Tue, 30 Nov 2021 14:27:24 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Subject: Re: [PATCH RFC 3/4] mm: percpu: Add generic pcpu_fc_alloc/free funciton Content-Language: en-US To: Dennis Zhou CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , References: <20211121093557.139034-1-wangkefeng.wang@huawei.com> <20211121093557.139034-4-wangkefeng.wang@huawei.com> From: Kefeng Wang In-Reply-To: X-Originating-IP: [10.174.177.243] X-ClientProxiedBy: dggeme704-chm.china.huawei.com (10.1.199.100) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211129_222729_958885_F3C1A771 X-CRM114-Status: GOOD ( 13.68 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Ck9uIDIwMjEvMTEvMzAgNjo0NSwgRGVubmlzIFpob3Ugd3JvdGU6Cj4gT24gU3VuLCBOb3YgMjEs IDIwMjEgYXQgMDU6MzU6NTZQTSArMDgwMCwgS2VmZW5nIFdhbmcgd3JvdGU6Cj4+IFdpdGggcHJl dmlvdXMgcGF0Y2gsIHdlIGNvdWxkIGFkZCBhIGdlbmVyaWMgcGNwdSBmaXJzdCBjaHVuawo+PiBh bGxvY2F0aW9uIGFuZCBmcmVlIGZ1bmN0aW9uIHRvIGNsZWFudXAgdGhlIGR1cGxpY2F0ZWQgZGVm aW5hdGlvbnMKPj4gb24gZWFjaCBhcmNoaXRlY3R1cmUuCj4+Cj4+IFNpZ25lZC1vZmYtYnk6IEtl ZmVuZyBXYW5nIDx3YW5na2VmZW5nLndhbmdAaHVhd2VpLmNvbT4KPj4gLS0tCj4+ICAgYXJjaC9t aXBzL21tL2luaXQuYyAgICAgICAgICAgIHwgMTYgKy0tLS0tLS0tCj4+ICAgYXJjaC9wb3dlcnBj L2tlcm5lbC9zZXR1cF82NC5jIHwgNTEgKystLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KPj4gICBh cmNoL3NwYXJjL2tlcm5lbC9zbXBfNjQuYyAgICAgfCA1MCArLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLQo+PiAgIGFyY2gveDg2L2tlcm5lbC9zZXR1cF9wZXJjcHUuYyB8IDU5ICstLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLQo+PiAgIGRyaXZlcnMvYmFzZS9hcmNoX251bWEuYyAgICAgICB8 IDE5ICstLS0tLS0tLS0KPj4gICBpbmNsdWRlL2xpbnV4L3BlcmNwdS5oICAgICAgICAgfCAgOSAr LS0tLQo+PiAgIG1tL3BlcmNwdS5jICAgICAgICAgICAgICAgICAgICB8IDY2ICsrKysrKysrKysr KysrKysrKy0tLS0tLS0tLS0tLS0tLS0KPj4gICA3IGZpbGVzIGNoYW5nZWQsIDQyIGluc2VydGlv bnMoKyksIDIyOCBkZWxldGlvbnMoLSkKLi4uCj4+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4 L3BlcmNwdS5oIGIvaW5jbHVkZS9saW51eC9wZXJjcHUuaAo+PiBpbmRleCA0MWJiNTQ3MTViMGMu LmQ3M2M5N2VmNGZmNCAxMDA2NDQKPj4gLS0tIGEvaW5jbHVkZS9saW51eC9wZXJjcHUuaAo+PiAr KysgYi9pbmNsdWRlL2xpbnV4L3BlcmNwdS5oCj4+IEBAIC05NSw5ICs5NSw2IEBAIGV4dGVybiBj b25zdCBjaGFyICogY29uc3QgcGNwdV9mY19uYW1lc1tQQ1BVX0ZDX05SXTsKPj4gICBleHRlcm4g ZW51bSBwY3B1X2ZjIHBjcHVfY2hvc2VuX2ZjOwo+PiAgIAo+PiAgIHR5cGVkZWYgaW50IChwY3B1 X2ZjX2NwdV90b19ub2RlX2ZuX3QpKGludCBjcHUpOwo+PiAtdHlwZWRlZiB2b2lkICogKCpwY3B1 X2ZjX2FsbG9jX2ZuX3QpKHVuc2lnbmVkIGludCBjcHUsIHNpemVfdCBzaXplLCBzaXplX3QgYWxp Z24sCj4+IC0JCQkJICAgICBwY3B1X2ZjX2NwdV90b19ub2RlX2ZuX3QgY3B1X3RvX25kX2ZuKTsK Pj4gLXR5cGVkZWYgdm9pZCAoKnBjcHVfZmNfZnJlZV9mbl90KSh2b2lkICpwdHIsIHNpemVfdCBz aXplKTsKPj4gICB0eXBlZGVmIHZvaWQgKCpwY3B1X2ZjX3BvcHVsYXRlX3B0ZV9mbl90KSh1bnNp Z25lZCBsb25nIGFkZHIpOwo+PiAgIHR5cGVkZWYgaW50IChwY3B1X2ZjX2NwdV9kaXN0YW5jZV9m bl90KSh1bnNpZ25lZCBpbnQgZnJvbSwgdW5zaWduZWQgaW50IHRvKTsKPj4gICAKPj4gQEAgLTEx MiwxNSArMTA5LDExIEBAIGV4dGVybiB2b2lkIF9faW5pdCBwY3B1X3NldHVwX2ZpcnN0X2NodW5r KGNvbnN0IHN0cnVjdCBwY3B1X2FsbG9jX2luZm8gKmFpLAo+PiAgIGV4dGVybiBpbnQgX19pbml0 IHBjcHVfZW1iZWRfZmlyc3RfY2h1bmsoc2l6ZV90IHJlc2VydmVkX3NpemUsIHNpemVfdCBkeW5f c2l6ZSwKPj4gICAJCQkJc2l6ZV90IGF0b21fc2l6ZSwKPj4gICAJCQkJcGNwdV9mY19jcHVfZGlz dGFuY2VfZm5fdCBjcHVfZGlzdGFuY2VfZm4sCj4+IC0JCQkJcGNwdV9mY19jcHVfdG9fbm9kZV9m bl90IGNwdV90b19uZF9mbiwKPj4gLQkJCQlwY3B1X2ZjX2FsbG9jX2ZuX3QgYWxsb2NfZm4sCj4+ IC0JCQkJcGNwdV9mY19mcmVlX2ZuX3QgZnJlZV9mbik7Cj4+ICsJCQkJcGNwdV9mY19jcHVfdG9f bm9kZV9mbl90IGNwdV90b19uZF9mbik7Cj4+ICAgI2VuZGlmCj4+ICAgCj4+ICAgI2lmZGVmIENP TkZJR19ORUVEX1BFUl9DUFVfUEFHRV9GSVJTVF9DSFVOSwo+PiAgIGV4dGVybiBpbnQgX19pbml0 IHBjcHVfcGFnZV9maXJzdF9jaHVuayhzaXplX3QgcmVzZXJ2ZWRfc2l6ZSwKPj4gLQkJCQlwY3B1 X2ZjX2FsbG9jX2ZuX3QgYWxsb2NfZm4sCj4+IC0JCQkJcGNwdV9mY19mcmVlX2ZuX3QgZnJlZV9m biwKPj4gICAJCQkJcGNwdV9mY19jcHVfdG9fbm9kZV9mbl90IGNwdV90b19uZF9mbiwKPj4gICAJ CQkJcGNwdV9mY19wb3B1bGF0ZV9wdGVfZm5fdCBwb3B1bGF0ZV9wdGVfZm4pOwo+PiAgICNlbmRp Zgo+PiBkaWZmIC0tZ2l0IGEvbW0vcGVyY3B1LmMgYi9tbS9wZXJjcHUuYwo+PiBpbmRleCAzZjZj ZjFmZjBiZTIuLmVmYWExY2JhZjczZCAxMDA2NDQKPj4gLS0tIGEvbW0vcGVyY3B1LmMKPj4gKysr IGIvbW0vcGVyY3B1LmMKPj4gQEAgLTI5OTIsNiArMjk5MiwzMCBAQCBzdGF0aWMgc3RydWN0IHBj cHVfYWxsb2NfaW5mbyAqIF9faW5pdCBfX2ZsYXR0ZW4gcGNwdV9idWlsZF9hbGxvY19pbmZvKAo+ PiAgIAo+PiAgIAlyZXR1cm4gYWk7Cj4+ICAgfQo+PiArCj4+ICtzdGF0aWMgdm9pZCAqIF9faW5p dCBwY3B1X2ZjX2FsbG9jKHVuc2lnbmVkIGludCBjcHUsIHNpemVfdCBzaXplLCBzaXplX3QgYWxp Z24sCj4+ICsJCQkJICAgcGNwdV9mY19jcHVfdG9fbm9kZV9mbl90IGNwdV90b19uZF9mbikKPj4g K3sKPj4gKwljb25zdCB1bnNpZ25lZCBsb25nIGdvYWwgPSBfX3BhKE1BWF9ETUFfQUREUkVTUyk7 Cj4+ICsJaW50IG5vZGUgPSBOVU1BX05PX05PREU7Cj4+ICsJdm9pZCAqcHRyOwo+PiArCj4+ICsJ aWYgKGNwdV90b19uZF9mbikKPj4gKwkJbm9kZSA9IGNwdV90b19uZF9mbihjcHUpOwo+PiArCj4+ ICsJaWYgKG5vZGUgPT0gTlVNQV9OT19OT0RFIHx8ICFub2RlX29ubGluZShub2RlKSB8fCAhTk9E RV9EQVRBKG5vZGUpKSB7Cj4+ICsJCXB0ciA9IG1lbWJsb2NrX2FsbG9jX2Zyb20oc2l6ZSwgYWxp Z24sIGdvYWwpOwo+PiArCX0gZWxzZSB7Cj4+ICsJCXB0ciA9IG1lbWJsb2NrX2FsbG9jX3RyeV9u aWQoc2l6ZSwgYWxpZ24sIGdvYWwsCj4+ICsJCQkJCSAgICAgTUVNQkxPQ0tfQUxMT0NfQUNDRVNT SUJMRSwgbm9kZSk7Cj4+ICsJfQo+PiArCXJldHVybiBwdHI7Cj4+ICt9Cj4gTXkgcHJlZmVyZW5j ZSBoZXJlIHdvdWxkIGJlIHRvIGtlZXAgdGhpcyBpZGVudGljYWwgdG8gdGhlIHg4Ngo+IGltcGxl bWVudGF0aW9uIHdoZXJlIHdlICNpZmRlZiBDT05GSUdfTlVNQS4KCkkgd2lsbCBhZGQgYmFjayAn I2lmZGVmIENPTkZJR19OVU1BJyzCoCBhbHNvIHdpbGwgYWRkIGJhY2sgdGhlIApwcl9kZWJ1Zy9w cl9pbmZvIHBhcnQgYXMgeDg2IGRvZXMsCgo+IE92ZXJhbGwgdGhpcyBtYWtlcyBzZW5zZS4KPgo+ IFRoYW5rcywKPiBEZW5uaXMKPiAuCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fXwpsaW51eC1yaXNjdiBtYWlsaW5nIGxpc3QKbGludXgtcmlzY3ZAbGlzdHMu aW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZv L2xpbnV4LXJpc2N2Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 362DCC433F5 for ; Tue, 30 Nov 2021 06:27:59 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4J3C2P5Jw8z3cZP for ; Tue, 30 Nov 2021 17:27:57 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=huawei.com (client-ip=45.249.212.255; helo=szxga08-in.huawei.com; envelope-from=wangkefeng.wang@huawei.com; receiver=) Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4J3C1w0N38z2xYG for ; Tue, 30 Nov 2021 17:27:31 +1100 (AEDT) Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4J3Byk5GSHz1DJpF; Tue, 30 Nov 2021 14:24:46 +0800 (CST) Received: from dggpemm500001.china.huawei.com (7.185.36.107) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 30 Nov 2021 14:27:26 +0800 Received: from [10.174.177.243] (10.174.177.243) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.20; Tue, 30 Nov 2021 14:27:25 +0800 Message-ID: Date: Tue, 30 Nov 2021 14:27:24 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Subject: Re: [PATCH RFC 3/4] mm: percpu: Add generic pcpu_fc_alloc/free funciton Content-Language: en-US To: Dennis Zhou References: <20211121093557.139034-1-wangkefeng.wang@huawei.com> <20211121093557.139034-4-wangkefeng.wang@huawei.com> From: Kefeng Wang In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.177.243] X-ClientProxiedBy: dggeme704-chm.china.huawei.com (10.1.199.100) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-ia64@vger.kernel.org, dave.hansen@linux.intel.com, linux-mips@vger.kernel.org, linux-mm@kvack.org, paulus@samba.org, hpa@zytor.com, sparclinux@vger.kernel.org, cl@linux.com, will@kernel.org, linux-riscv@lists.infradead.org, x86@kernel.org, mingo@redhat.com, catalin.marinas@arm.com, aou@eecs.berkeley.edu, bp@alien8.de, paul.walmsley@sifive.com, tglx@linutronix.de, linux-arm-kernel@lists.infradead.org, tsbogend@alpha.franken.de, gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, palmer@dabbelt.com, tj@kernel.org, akpm@linux-foundation.org, linuxppc-dev@lists.ozlabs.org, davem@davemloft.net Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On 2021/11/30 6:45, Dennis Zhou wrote: > On Sun, Nov 21, 2021 at 05:35:56PM +0800, Kefeng Wang wrote: >> With previous patch, we could add a generic pcpu first chunk >> allocation and free function to cleanup the duplicated definations >> on each architecture. >> >> Signed-off-by: Kefeng Wang >> --- >> arch/mips/mm/init.c | 16 +-------- >> arch/powerpc/kernel/setup_64.c | 51 ++------------------------ >> arch/sparc/kernel/smp_64.c | 50 +------------------------- >> arch/x86/kernel/setup_percpu.c | 59 +----------------------------- >> drivers/base/arch_numa.c | 19 +--------- >> include/linux/percpu.h | 9 +---- >> mm/percpu.c | 66 ++++++++++++++++++---------------- >> 7 files changed, 42 insertions(+), 228 deletions(-) ... >> diff --git a/include/linux/percpu.h b/include/linux/percpu.h >> index 41bb54715b0c..d73c97ef4ff4 100644 >> --- a/include/linux/percpu.h >> +++ b/include/linux/percpu.h >> @@ -95,9 +95,6 @@ extern const char * const pcpu_fc_names[PCPU_FC_NR]; >> extern enum pcpu_fc pcpu_chosen_fc; >> >> typedef int (pcpu_fc_cpu_to_node_fn_t)(int cpu); >> -typedef void * (*pcpu_fc_alloc_fn_t)(unsigned int cpu, size_t size, size_t align, >> - pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn); >> -typedef void (*pcpu_fc_free_fn_t)(void *ptr, size_t size); >> typedef void (*pcpu_fc_populate_pte_fn_t)(unsigned long addr); >> typedef int (pcpu_fc_cpu_distance_fn_t)(unsigned int from, unsigned int to); >> >> @@ -112,15 +109,11 @@ extern void __init pcpu_setup_first_chunk(const struct pcpu_alloc_info *ai, >> extern int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size, >> size_t atom_size, >> pcpu_fc_cpu_distance_fn_t cpu_distance_fn, >> - pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn, >> - pcpu_fc_alloc_fn_t alloc_fn, >> - pcpu_fc_free_fn_t free_fn); >> + pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn); >> #endif >> >> #ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK >> extern int __init pcpu_page_first_chunk(size_t reserved_size, >> - pcpu_fc_alloc_fn_t alloc_fn, >> - pcpu_fc_free_fn_t free_fn, >> pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn, >> pcpu_fc_populate_pte_fn_t populate_pte_fn); >> #endif >> diff --git a/mm/percpu.c b/mm/percpu.c >> index 3f6cf1ff0be2..efaa1cbaf73d 100644 >> --- a/mm/percpu.c >> +++ b/mm/percpu.c >> @@ -2992,6 +2992,30 @@ static struct pcpu_alloc_info * __init __flatten pcpu_build_alloc_info( >> >> return ai; >> } >> + >> +static void * __init pcpu_fc_alloc(unsigned int cpu, size_t size, size_t align, >> + pcpu_fc_cpu_to_node_fn_t cpu_to_nd_fn) >> +{ >> + const unsigned long goal = __pa(MAX_DMA_ADDRESS); >> + int node = NUMA_NO_NODE; >> + void *ptr; >> + >> + if (cpu_to_nd_fn) >> + node = cpu_to_nd_fn(cpu); >> + >> + if (node == NUMA_NO_NODE || !node_online(node) || !NODE_DATA(node)) { >> + ptr = memblock_alloc_from(size, align, goal); >> + } else { >> + ptr = memblock_alloc_try_nid(size, align, goal, >> + MEMBLOCK_ALLOC_ACCESSIBLE, node); >> + } >> + return ptr; >> +} > My preference here would be to keep this identical to the x86 > implementation where we #ifdef CONFIG_NUMA. I will add back '#ifdef CONFIG_NUMA',  also will add back the pr_debug/pr_info part as x86 does, > Overall this makes sense. > > Thanks, > Dennis > . From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0A04EC433EF for ; Tue, 30 Nov 2021 06:28:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:CC:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Y5WkN6XWtyRuRvFS0Q68FEG0MU9NAyz6RNFnYJd6pL4=; b=x+UOCxDDwYYalG R0W9vsM1kmzHizn9CQqC/3g0V4DoUUtA8a5MC+g/fu45zfqYOh+861zbEFhOdSvb4G21ooVxvAQJA EJ2N94xV/b30CLTO5Du4kWO8OPJlthxXglvL8DNEObETEeKk8w+pVUwEIfrrHID8ADwpvskTKUshD 3Imy6ecJzxSR2VMQaivuT61qVcfwcZUTtYm8xtfNhfBQ6yAiqgMGme4BnGFAIkjs0XEv+MfUHgQC9 NNIvwdiff34KAvTSqkw7XX6H91bh1TfCB7fJjr+Kkhrl84TZ3zWCRm8KxQDmQu8N7A3q7cgEXazi9 3urj+JxlOdXSp4xU0xiQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mrwc1-003nhh-4A; Tue, 30 Nov 2021 06:27:33 +0000 Received: from szxga08-in.huawei.com ([45.249.212.255]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mrwbx-003nge-IK; Tue, 30 Nov 2021 06:27:31 +0000 Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4J3Byk5GSHz1DJpF; Tue, 30 Nov 2021 14:24:46 +0800 (CST) Received: from dggpemm500001.china.huawei.com (7.185.36.107) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 30 Nov 2021 14:27:26 +0800 Received: from [10.174.177.243] (10.174.177.243) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.20; Tue, 30 Nov 2021 14:27:25 +0800 Message-ID: Date: Tue, 30 Nov 2021 14:27:24 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Subject: Re: [PATCH RFC 3/4] mm: percpu: Add generic pcpu_fc_alloc/free funciton Content-Language: en-US To: Dennis Zhou CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , References: <20211121093557.139034-1-wangkefeng.wang@huawei.com> <20211121093557.139034-4-wangkefeng.wang@huawei.com> From: Kefeng Wang In-Reply-To: X-Originating-IP: [10.174.177.243] X-ClientProxiedBy: dggeme704-chm.china.huawei.com (10.1.199.100) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211129_222729_958885_F3C1A771 X-CRM114-Status: GOOD ( 13.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Ck9uIDIwMjEvMTEvMzAgNjo0NSwgRGVubmlzIFpob3Ugd3JvdGU6Cj4gT24gU3VuLCBOb3YgMjEs IDIwMjEgYXQgMDU6MzU6NTZQTSArMDgwMCwgS2VmZW5nIFdhbmcgd3JvdGU6Cj4+IFdpdGggcHJl dmlvdXMgcGF0Y2gsIHdlIGNvdWxkIGFkZCBhIGdlbmVyaWMgcGNwdSBmaXJzdCBjaHVuawo+PiBh bGxvY2F0aW9uIGFuZCBmcmVlIGZ1bmN0aW9uIHRvIGNsZWFudXAgdGhlIGR1cGxpY2F0ZWQgZGVm aW5hdGlvbnMKPj4gb24gZWFjaCBhcmNoaXRlY3R1cmUuCj4+Cj4+IFNpZ25lZC1vZmYtYnk6IEtl ZmVuZyBXYW5nIDx3YW5na2VmZW5nLndhbmdAaHVhd2VpLmNvbT4KPj4gLS0tCj4+ICAgYXJjaC9t aXBzL21tL2luaXQuYyAgICAgICAgICAgIHwgMTYgKy0tLS0tLS0tCj4+ICAgYXJjaC9wb3dlcnBj L2tlcm5lbC9zZXR1cF82NC5jIHwgNTEgKystLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KPj4gICBh cmNoL3NwYXJjL2tlcm5lbC9zbXBfNjQuYyAgICAgfCA1MCArLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLQo+PiAgIGFyY2gveDg2L2tlcm5lbC9zZXR1cF9wZXJjcHUuYyB8IDU5ICstLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLQo+PiAgIGRyaXZlcnMvYmFzZS9hcmNoX251bWEuYyAgICAgICB8 IDE5ICstLS0tLS0tLS0KPj4gICBpbmNsdWRlL2xpbnV4L3BlcmNwdS5oICAgICAgICAgfCAgOSAr LS0tLQo+PiAgIG1tL3BlcmNwdS5jICAgICAgICAgICAgICAgICAgICB8IDY2ICsrKysrKysrKysr KysrKysrKy0tLS0tLS0tLS0tLS0tLS0KPj4gICA3IGZpbGVzIGNoYW5nZWQsIDQyIGluc2VydGlv bnMoKyksIDIyOCBkZWxldGlvbnMoLSkKLi4uCj4+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4 L3BlcmNwdS5oIGIvaW5jbHVkZS9saW51eC9wZXJjcHUuaAo+PiBpbmRleCA0MWJiNTQ3MTViMGMu LmQ3M2M5N2VmNGZmNCAxMDA2NDQKPj4gLS0tIGEvaW5jbHVkZS9saW51eC9wZXJjcHUuaAo+PiAr KysgYi9pbmNsdWRlL2xpbnV4L3BlcmNwdS5oCj4+IEBAIC05NSw5ICs5NSw2IEBAIGV4dGVybiBj b25zdCBjaGFyICogY29uc3QgcGNwdV9mY19uYW1lc1tQQ1BVX0ZDX05SXTsKPj4gICBleHRlcm4g ZW51bSBwY3B1X2ZjIHBjcHVfY2hvc2VuX2ZjOwo+PiAgIAo+PiAgIHR5cGVkZWYgaW50IChwY3B1 X2ZjX2NwdV90b19ub2RlX2ZuX3QpKGludCBjcHUpOwo+PiAtdHlwZWRlZiB2b2lkICogKCpwY3B1 X2ZjX2FsbG9jX2ZuX3QpKHVuc2lnbmVkIGludCBjcHUsIHNpemVfdCBzaXplLCBzaXplX3QgYWxp Z24sCj4+IC0JCQkJICAgICBwY3B1X2ZjX2NwdV90b19ub2RlX2ZuX3QgY3B1X3RvX25kX2ZuKTsK Pj4gLXR5cGVkZWYgdm9pZCAoKnBjcHVfZmNfZnJlZV9mbl90KSh2b2lkICpwdHIsIHNpemVfdCBz aXplKTsKPj4gICB0eXBlZGVmIHZvaWQgKCpwY3B1X2ZjX3BvcHVsYXRlX3B0ZV9mbl90KSh1bnNp Z25lZCBsb25nIGFkZHIpOwo+PiAgIHR5cGVkZWYgaW50IChwY3B1X2ZjX2NwdV9kaXN0YW5jZV9m bl90KSh1bnNpZ25lZCBpbnQgZnJvbSwgdW5zaWduZWQgaW50IHRvKTsKPj4gICAKPj4gQEAgLTEx MiwxNSArMTA5LDExIEBAIGV4dGVybiB2b2lkIF9faW5pdCBwY3B1X3NldHVwX2ZpcnN0X2NodW5r KGNvbnN0IHN0cnVjdCBwY3B1X2FsbG9jX2luZm8gKmFpLAo+PiAgIGV4dGVybiBpbnQgX19pbml0 IHBjcHVfZW1iZWRfZmlyc3RfY2h1bmsoc2l6ZV90IHJlc2VydmVkX3NpemUsIHNpemVfdCBkeW5f c2l6ZSwKPj4gICAJCQkJc2l6ZV90IGF0b21fc2l6ZSwKPj4gICAJCQkJcGNwdV9mY19jcHVfZGlz dGFuY2VfZm5fdCBjcHVfZGlzdGFuY2VfZm4sCj4+IC0JCQkJcGNwdV9mY19jcHVfdG9fbm9kZV9m bl90IGNwdV90b19uZF9mbiwKPj4gLQkJCQlwY3B1X2ZjX2FsbG9jX2ZuX3QgYWxsb2NfZm4sCj4+ IC0JCQkJcGNwdV9mY19mcmVlX2ZuX3QgZnJlZV9mbik7Cj4+ICsJCQkJcGNwdV9mY19jcHVfdG9f bm9kZV9mbl90IGNwdV90b19uZF9mbik7Cj4+ICAgI2VuZGlmCj4+ICAgCj4+ICAgI2lmZGVmIENP TkZJR19ORUVEX1BFUl9DUFVfUEFHRV9GSVJTVF9DSFVOSwo+PiAgIGV4dGVybiBpbnQgX19pbml0 IHBjcHVfcGFnZV9maXJzdF9jaHVuayhzaXplX3QgcmVzZXJ2ZWRfc2l6ZSwKPj4gLQkJCQlwY3B1 X2ZjX2FsbG9jX2ZuX3QgYWxsb2NfZm4sCj4+IC0JCQkJcGNwdV9mY19mcmVlX2ZuX3QgZnJlZV9m biwKPj4gICAJCQkJcGNwdV9mY19jcHVfdG9fbm9kZV9mbl90IGNwdV90b19uZF9mbiwKPj4gICAJ CQkJcGNwdV9mY19wb3B1bGF0ZV9wdGVfZm5fdCBwb3B1bGF0ZV9wdGVfZm4pOwo+PiAgICNlbmRp Zgo+PiBkaWZmIC0tZ2l0IGEvbW0vcGVyY3B1LmMgYi9tbS9wZXJjcHUuYwo+PiBpbmRleCAzZjZj ZjFmZjBiZTIuLmVmYWExY2JhZjczZCAxMDA2NDQKPj4gLS0tIGEvbW0vcGVyY3B1LmMKPj4gKysr IGIvbW0vcGVyY3B1LmMKPj4gQEAgLTI5OTIsNiArMjk5MiwzMCBAQCBzdGF0aWMgc3RydWN0IHBj cHVfYWxsb2NfaW5mbyAqIF9faW5pdCBfX2ZsYXR0ZW4gcGNwdV9idWlsZF9hbGxvY19pbmZvKAo+ PiAgIAo+PiAgIAlyZXR1cm4gYWk7Cj4+ICAgfQo+PiArCj4+ICtzdGF0aWMgdm9pZCAqIF9faW5p dCBwY3B1X2ZjX2FsbG9jKHVuc2lnbmVkIGludCBjcHUsIHNpemVfdCBzaXplLCBzaXplX3QgYWxp Z24sCj4+ICsJCQkJICAgcGNwdV9mY19jcHVfdG9fbm9kZV9mbl90IGNwdV90b19uZF9mbikKPj4g K3sKPj4gKwljb25zdCB1bnNpZ25lZCBsb25nIGdvYWwgPSBfX3BhKE1BWF9ETUFfQUREUkVTUyk7 Cj4+ICsJaW50IG5vZGUgPSBOVU1BX05PX05PREU7Cj4+ICsJdm9pZCAqcHRyOwo+PiArCj4+ICsJ aWYgKGNwdV90b19uZF9mbikKPj4gKwkJbm9kZSA9IGNwdV90b19uZF9mbihjcHUpOwo+PiArCj4+ ICsJaWYgKG5vZGUgPT0gTlVNQV9OT19OT0RFIHx8ICFub2RlX29ubGluZShub2RlKSB8fCAhTk9E RV9EQVRBKG5vZGUpKSB7Cj4+ICsJCXB0ciA9IG1lbWJsb2NrX2FsbG9jX2Zyb20oc2l6ZSwgYWxp Z24sIGdvYWwpOwo+PiArCX0gZWxzZSB7Cj4+ICsJCXB0ciA9IG1lbWJsb2NrX2FsbG9jX3RyeV9u aWQoc2l6ZSwgYWxpZ24sIGdvYWwsCj4+ICsJCQkJCSAgICAgTUVNQkxPQ0tfQUxMT0NfQUNDRVNT SUJMRSwgbm9kZSk7Cj4+ICsJfQo+PiArCXJldHVybiBwdHI7Cj4+ICt9Cj4gTXkgcHJlZmVyZW5j ZSBoZXJlIHdvdWxkIGJlIHRvIGtlZXAgdGhpcyBpZGVudGljYWwgdG8gdGhlIHg4Ngo+IGltcGxl bWVudGF0aW9uIHdoZXJlIHdlICNpZmRlZiBDT05GSUdfTlVNQS4KCkkgd2lsbCBhZGQgYmFjayAn I2lmZGVmIENPTkZJR19OVU1BJyzCoCBhbHNvIHdpbGwgYWRkIGJhY2sgdGhlIApwcl9kZWJ1Zy9w cl9pbmZvIHBhcnQgYXMgeDg2IGRvZXMsCgo+IE92ZXJhbGwgdGhpcyBtYWtlcyBzZW5zZS4KPgo+ IFRoYW5rcywKPiBEZW5uaXMKPiAuCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2Vy bmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1h bi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==