From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 183B5C433E0 for ; Wed, 20 Jan 2021 10:59:53 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B5B3722242 for ; Wed, 20 Jan 2021 10:59:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5B3722242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-ID:References:In-Reply-To:Subject:To:From: Date:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NkW/03XMgfbORzx+zOow0kE0J+7vJllidHVucWyda70=; b=AwNd/IfcwsIHSnORwdw8OXgee aAGfPgSDivUer280IvVtlcg65dRB1UFYe6RqBdpuzgE+zNcjoSBJRmFwWjW6ZFNzHQ1TM5DsRNp6D QUxCEg+fUj32mSl4HlaRdUCaKMeL6nY+4/XSDa/5Ea1FXPI9GHihftO4nPLnqc0VURZIhZ40JZB/L C8hqT2Mwy9p7nXbO2LbDCCQYs+4jXPvB9yQMKWjhb1e5w1gOlZeu1bS1nNcXWrNJPgMYikZ8/Myyh rYPM3jPIORorKl1wIucSSS98BVwS47rtppeFECqFmU8wsWb2orTQKYSa2rd/ODEM/vrNEA1QZ3Awe vJGga+Lng==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2BC2-0004Eg-0L; Wed, 20 Jan 2021 10:58:30 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2BBy-0004Ds-2b for linux-arm-kernel@lists.infradead.org; Wed, 20 Jan 2021 10:58:26 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4921C21744; Wed, 20 Jan 2021 10:58:24 +0000 (UTC) Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94) (envelope-from ) id 1l2BBu-008vIz-4y; Wed, 20 Jan 2021 10:58:22 +0000 MIME-Version: 1.0 Date: Wed, 20 Jan 2021 10:58:22 +0000 From: Marc Zyngier To: Justin He Subject: Re: [RFC PATCH 0/2] Avoid booting stall caused by idmap_kpti_install_ng_mappings In-Reply-To: References: User-Agent: Roundcube Webmail/1.4.10 Message-ID: X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: Justin.He@arm.com, Catalin.Marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Anshuman.Khandual@arm.com, Suzuki.Poulose@arm.com, Mark.Rutland@arm.com, gustavoars@kernel.org, richard.henderson@linaro.org, Dave.Martin@arm.com, Steven.Price@arm.com, akpm@linux-foundation.org, rppt@kernel.org, ardb@kernel.org, gshan@redhat.com, wangkefeng.wang@huawei.com, broonie@kernel.org, Cristian.Marussi@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210120_055826_267171_D6DD1DA7 X-CRM114-Status: GOOD ( 15.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kefeng Wang , Gavin Shan , Suzuki Poulose , Ard Biesheuvel , Catalin Marinas , Anshuman Khandual , Richard Henderson , linux-kernel@vger.kernel.org, "Gustavo A. R. Silva" , Mark Brown , Cristian Marussi , Mike Rapoport , Andrew Morton , Steven Price , Will Deacon , Dave P Martin , linux-arm-kernel@lists.infradead.org Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgSnVzdGluLAoKT24gMjAyMS0wMS0yMCAwNDo1MSwgSnVzdGluIEhlIHdyb3RlOgo+IEhpLAo+ IEtpbmRseSBwaW5nIPCfmIoKPiAKPj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0KPj4gRnJv bTogSmlhIEhlIDxqdXN0aW4uaGVAYXJtLmNvbT4KPj4gU2VudDogV2VkbmVzZGF5LCBKYW51YXJ5 IDEzLCAyMDIxIDk6NDEgQU0KPj4gVG86IENhdGFsaW4gTWFyaW5hcyA8Q2F0YWxpbi5NYXJpbmFz QGFybS5jb20+OyBXaWxsIERlYWNvbgo+PiA8d2lsbEBrZXJuZWwub3JnPjsgbGludXgtYXJtLWtl cm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnOyBsaW51eC0KPj4ga2VybmVsQHZnZXIua2VybmVsLm9y Zwo+PiBDYzogQW5zaHVtYW4gS2hhbmR1YWwgPEFuc2h1bWFuLktoYW5kdWFsQGFybS5jb20+OyBT dXp1a2kgUG91bG9zZQo+PiA8U3V6dWtpLlBvdWxvc2VAYXJtLmNvbT47IEp1c3RpbiBIZSA8SnVz dGluLkhlQGFybS5jb20+OyBNYXJrIFJ1dGxhbmQKPj4gPE1hcmsuUnV0bGFuZEBhcm0uY29tPjsg R3VzdGF2byBBLiBSLiBTaWx2YSA8Z3VzdGF2b2Fyc0BrZXJuZWwub3JnPjsKPj4gUmljaGFyZCBI ZW5kZXJzb24gPHJpY2hhcmQuaGVuZGVyc29uQGxpbmFyby5vcmc+OyBEYXZlIFAgTWFydGluCj4+ IDxEYXZlLk1hcnRpbkBhcm0uY29tPjsgU3RldmVuIFByaWNlIDxTdGV2ZW4uUHJpY2VAYXJtLmNv bT47IEFuZHJldyAKPj4gTW9ydG9uCj4+IDxha3BtQGxpbnV4LWZvdW5kYXRpb24ub3JnPjsgTWlr ZSBSYXBvcG9ydCA8cnBwdEBrZXJuZWwub3JnPjsgQXJkCj4+IEJpZXNoZXV2ZWwgPGFyZGJAa2Vy bmVsLm9yZz47IEdhdmluIFNoYW4gPGdzaGFuQHJlZGhhdC5jb20+OyBLZWZlbmcgCj4+IFdhbmcK Pj4gPHdhbmdrZWZlbmcud2FuZ0BodWF3ZWkuY29tPjsgTWFyayBCcm93biA8YnJvb25pZUBrZXJu ZWwub3JnPjsgTWFyYyAKPj4gWnluZ2llcgo+PiA8bWF6QGtlcm5lbC5vcmc+OyBDcmlzdGlhbiBN YXJ1c3NpIDxDcmlzdGlhbi5NYXJ1c3NpQGFybS5jb20+Cj4+IFN1YmplY3Q6IFtSRkMgUEFUQ0gg MC8yXSBBdm9pZCBib290aW5nIHN0YWxsIGNhdXNlZCBieQo+PiAKPj4gVGhlcmUgaXMgYSAxMHMg c3RhbGwgaW4gaWRtYXBfa3B0aV9pbnN0YWxsX25nX21hcHBpbmdzIHdoZW4ga2VybmVsIAo+PiBi b290cwo+PiBvbiBhIEFtcGVyZSBFTUFHIHNlcnZlci4KPj4gCj4+IENvbW1pdCBmOTkyYjRkZmQ1 OGIgKCJhcm02NDoga3B0aTogQWRkIC0+ZW5hYmxlIGNhbGxiYWNrIHRvIHJlbWFwCj4+IHN3YXBw ZXIgdXNpbmcgbkcgbWFwcGluZ3MiKSB1cGRhdGVzIHRoZSBuRyBiaXQgcnVudGltZSBpZiBrcHRp IGlzCj4+IHJlcXVpcmVkLgo+PiAKPj4gQnV0IHRoaW5ncyBnZXQgd29yc2UgaWYgcm9kYXRhPWZ1 bGwgaW4gbWFwX21lbSgpLiBOT19CTE9DS19NQVBQSU5HUyB8Cj4+IE5PX0NPTlRfTUFQUElOR1Mg aXMgcmVxdWlyZWQgd2hlbiBjcmVhdGluZyBwYWdldGFibGUgbWFwcGluZy4gSGVuY2UgCj4+IGFs bAo+PiBwdGVzIGFyZSBmdWxseSBtYXBwZWQgaW4gdGhpcyBjYXNlLiBPbiBhIEFtcGVyZSBFTUFH IHNlcnZlciB3aXRoIDI1NkcKPj4gbWVtb3J5KHBhZ2VzaXplPTRrKSwgaXQgY2F1c2VzIHRoZSAx MHMgc3RhbGwuCj4+IAo+PiBBZnRlciBtb3ZpbmcgaW5pdF9jcHVfZmVhdHVyZXMoKSBhaGVhZCBv ZiBlYXJseV9maXhtYXBfaW5pdCgpLCB3ZSBjYW4gCj4+IHVzZQo+PiBjcHVfaGF2ZV9jb25zdF9j YXAgZWFybGllciB0aGFuIGJlZm9yZS4gSGVuY2Ugd2UgY2FuIGF2b2lkIHRoaXMgc3RhbGwKPj4g YnkgdXBkYXRpbmcgYXJtNjRfdXNlX25nX21hcHBpbmdzLgo+PiAKPj4gQWZ0ZXIgdGhpcyBwYXRj aCBzZXJpZXMsIGl0IHJlZHVjZXMgdGhlIGtlcm5lbCBib290IHRpbWUgZnJvbSAxNC43cyB0bwo+ PiA0LjFzOgo+PiBCZWZvcmU6Cj4+IFsgICAxNC43NTc1NjldIEZyZWVpbmcgaW5pdHJkIG1lbW9y eTogNjA3NTJLCj4+IEFmdGVyOgo+PiBbICAgIDQuMTM4ODE5XSBGcmVlaW5nIGluaXRyZCBtZW1v cnk6IDYwNzUySwo+PiAKPj4gU2V0IGl0IGFzIFJGQyBiZWNhdXNlIEkgd2FudCB0byByZXNvbHZl IGFueSBvdGhlciBwb2ludHMgd2hpY2ggSSBoYXZlCj4+IG1pc2NvbmVybmVkLgoKQnV0IHlvdSBk b24ndCByZWFsbHkgZXhwbGFpbiAqd2h5KiBoYXZpbmcgdGhlIENQVSBGZWF0dXJlIGRpc2NvdmVy eQplYXJseSBoZWxwcyBhdCBhbGwuIElzIHRoYXQgc28gdGhhdCB5b3UgY2FuIGJ5cGFzcyB0aGUg aWRtYXAgbWFwcGluZz8KSSdkIGV4cGVjdCBzb21ldGhpbmcgdGhhdCBleHBsYWluIHRoZSBwcm9i bGVtIGluc3RlYWQgb2YgcGFyYXBocmFzaW5nCnRoZSBwYXRjaGVzLgoKQW5vdGhlciB0aGluZyBp cyB3aGV0aGVyIHlvdSBoYXZlIHRlc3RlZCB0aGlzIG9uIHNvbWUgVGh1bmRlclggSFcKKHRoZSBm aXJzdCB2ZXJzaW9uLCBub3QgVFgyKSwgYXMgdGhpcyBpcyB0aGUgd2hvbGUgcmVhc29uIGZvciB0 aGlzCmNvZGUuLi4KClRoYW5rcywKCiAgICAgICAgIE0uCi0tIApKYXp6IGlzIG5vdCBkZWFkLiBJ dCBqdXN0IHNtZWxscyBmdW5ueS4uLgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtl cm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxt YW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 721A1C10DCE for ; Wed, 20 Jan 2021 11:51:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 387B6206EC for ; Wed, 20 Jan 2021 11:51:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733102AbhATLkN (ORCPT ); Wed, 20 Jan 2021 06:40:13 -0500 Received: from mail.kernel.org ([198.145.29.99]:55386 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388574AbhATK7E (ORCPT ); Wed, 20 Jan 2021 05:59:04 -0500 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4921C21744; Wed, 20 Jan 2021 10:58:24 +0000 (UTC) Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94) (envelope-from ) id 1l2BBu-008vIz-4y; Wed, 20 Jan 2021 10:58:22 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 20 Jan 2021 10:58:22 +0000 From: Marc Zyngier To: Justin He Cc: Catalin Marinas , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Anshuman Khandual , Suzuki Poulose , Mark Rutland , "Gustavo A. R. Silva" , Richard Henderson , Dave P Martin , Steven Price , Andrew Morton , Mike Rapoport , Ard Biesheuvel , Gavin Shan , Kefeng Wang , Mark Brown , Cristian Marussi Subject: Re: [RFC PATCH 0/2] Avoid booting stall caused by idmap_kpti_install_ng_mappings In-Reply-To: References: User-Agent: Roundcube Webmail/1.4.10 Message-ID: X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: Justin.He@arm.com, Catalin.Marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Anshuman.Khandual@arm.com, Suzuki.Poulose@arm.com, Mark.Rutland@arm.com, gustavoars@kernel.org, richard.henderson@linaro.org, Dave.Martin@arm.com, Steven.Price@arm.com, akpm@linux-foundation.org, rppt@kernel.org, ardb@kernel.org, gshan@redhat.com, wangkefeng.wang@huawei.com, broonie@kernel.org, Cristian.Marussi@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Justin, On 2021-01-20 04:51, Justin He wrote: > Hi, > Kindly ping 😊 > >> -----Original Message----- >> From: Jia He >> Sent: Wednesday, January 13, 2021 9:41 AM >> To: Catalin Marinas ; Will Deacon >> ; linux-arm-kernel@lists.infradead.org; linux- >> kernel@vger.kernel.org >> Cc: Anshuman Khandual ; Suzuki Poulose >> ; Justin He ; Mark Rutland >> ; Gustavo A. R. Silva ; >> Richard Henderson ; Dave P Martin >> ; Steven Price ; Andrew >> Morton >> ; Mike Rapoport ; Ard >> Biesheuvel ; Gavin Shan ; Kefeng >> Wang >> ; Mark Brown ; Marc >> Zyngier >> ; Cristian Marussi >> Subject: [RFC PATCH 0/2] Avoid booting stall caused by >> >> There is a 10s stall in idmap_kpti_install_ng_mappings when kernel >> boots >> on a Ampere EMAG server. >> >> Commit f992b4dfd58b ("arm64: kpti: Add ->enable callback to remap >> swapper using nG mappings") updates the nG bit runtime if kpti is >> required. >> >> But things get worse if rodata=full in map_mem(). NO_BLOCK_MAPPINGS | >> NO_CONT_MAPPINGS is required when creating pagetable mapping. Hence >> all >> ptes are fully mapped in this case. On a Ampere EMAG server with 256G >> memory(pagesize=4k), it causes the 10s stall. >> >> After moving init_cpu_features() ahead of early_fixmap_init(), we can >> use >> cpu_have_const_cap earlier than before. Hence we can avoid this stall >> by updating arm64_use_ng_mappings. >> >> After this patch series, it reduces the kernel boot time from 14.7s to >> 4.1s: >> Before: >> [ 14.757569] Freeing initrd memory: 60752K >> After: >> [ 4.138819] Freeing initrd memory: 60752K >> >> Set it as RFC because I want to resolve any other points which I have >> misconerned. But you don't really explain *why* having the CPU Feature discovery early helps at all. Is that so that you can bypass the idmap mapping? I'd expect something that explain the problem instead of paraphrasing the patches. Another thing is whether you have tested this on some ThunderX HW (the first version, not TX2), as this is the whole reason for this code... Thanks, M. -- Jazz is not dead. It just smells funny...