From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA8B3C433EF for ; Fri, 4 Mar 2022 17:59:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/h+yV8mCRN6XutV4pt9nowV3ZzHpter+KoTq2ja/roc=; b=lY4+o7vJsdlq0+ IviupoKDGK4urkyZ8gma2VS0NyoDFg/CIUSDXbc9j9tYnQp3osLp2D9NXveeR+IZVfhuG0DPzW4cr h9mhPRzJp0r0LBfg+wS7cRaVIgznTflHox6nsdfDgxCsz8diy1f1FFB0E+blLe5FfNyHR4N95MF8h MfP7+GTLkwMP8amlubeZpA4EoXlcCMmGg+ENGC4j6atnbBHqMEWFYXwShjLve70Y5muNfqt+GmfMT HwaLUcrsnuSHko5T24c8EZw2Pi4Rr657pxk6bKpjlHww/Vk9oDxcN4i0cTEmhgGMU9OiQxr/gvDXR +2M9HtTFzgERms39sKCQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nQCCB-00BT7K-MR; Fri, 04 Mar 2022 17:58:27 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nQCBU-00BSrj-Bt for linux-arm-kernel@lists.infradead.org; Fri, 04 Mar 2022 17:57:46 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EBB3060B6F; Fri, 4 Mar 2022 17:57:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 06CD4C340F1; Fri, 4 Mar 2022 17:57:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646416663; bh=SHbmU2J96MyeCvYogtj9qosC5y/D5HuZ1lwMXtZ62aM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ma3RZ5f+mpnrFtMTRhVvMP8DWxJCjDM2fcT80SBVAQMrwzTzlTa5ot3QGMnJFjTHH RKF2QW0UVqagavHPwzsXIGWaXn5NfXwgP5wu1uQFgWmt9ftjxDPkqQOhE4IOhir4jS 9Tv1aN9GJB7+Z1qZLEprA7aSdLw8yYDslvLUrTW0YpqiTxukfBkVSu9wyAf8Y/42MM p9ldDvlrOoBePGVUVk8oSEQj7GkAv4RxzxAz4JQzmTcNqWVLWdSJMBLGwK3hPVWkQy L3zLAdPlcKDFADUXvoGzUbeP5KWjTuI/7ctb0f9y0gocyanpKfI5JM1Shd4/oilpuc lhGSAfoYP06lQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: mark.rutland@arm.com, android-kvm@google.com, Ard Biesheuvel , Marc Zyngier , Will Deacon Subject: [RFC PATCH 4/8] arm64: head: avoid cache invalidation when entering with the MMU on Date: Fri, 4 Mar 2022 18:56:53 +0100 Message-Id: <20220304175657.2744400-5-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220304175657.2744400-1-ardb@kernel.org> References: <20220304175657.2744400-1-ardb@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220304_095744_527452_003A1D0B X-CRM114-Status: GOOD ( 13.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The primary entry code populates memory with the MMU and caches disabled, and therefore needs to go out of its way to prevent dirty but stale cachelines from potentially corrupting these memory contents inadvertently. When entering with the MMU on, this is not needed, so skip it. While at it, renumber some asm labels to avoid confusion. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 23 ++++++++++++++------ 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 0726ce0d6fd4..b82c86fc9141 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -149,11 +149,13 @@ SYM_CODE_START_LOCAL(preserve_boot_args) stp x21, x1, [x0] // x0 .. x3 at kernel entry stp x2, x3, [x0, #16] + cbnz x25, 0f // skip cache invalidation if MMU is on dmb sy // needed before dc ivac with // MMU off add x1, x0, #0x20 // 4 x 8 bytes b dcache_inval_poc // tail call +0: ret SYM_CODE_END(preserve_boot_args) /* @@ -296,6 +298,8 @@ SYM_CODE_END(preserve_boot_args) SYM_FUNC_START_LOCAL(__create_page_tables) mov x28, lr + cbnz x25, 0f // skip cache invalidation if MMU is on + /* * Invalidate the init page tables to avoid potential dirty cache lines * being evicted. Other page tables are allocated in rodata as part of @@ -309,7 +313,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables) /* * Clear the init page tables. */ - adrp x0, init_pg_dir +0: adrp x0, init_pg_dir adrp x1, init_pg_end sub x1, x1, x0 1: stp xzr, xzr, [x0], #16 @@ -331,15 +335,16 @@ SYM_FUNC_START_LOCAL(__create_page_tables) mrs_s x6, SYS_ID_AA64MMFR2_EL1 and x6, x6, #(0xf << ID_AA64MMFR2_LVA_SHIFT) mov x5, #52 - cbnz x6, 1f + cbnz x6, 2f #endif mov x5, #VA_BITS_MIN -1: +2: adr_l x6, vabits_actual str x5, [x6] + cbnz x25, 3f // skip cache invalidation if MMU is on dmb sy dc ivac, x6 // Invalidate potentially stale cache line - +3: /* * VA_BITS may be too small to allow for an ID mapping to be created * that covers system RAM if that is located sufficiently high in the @@ -355,12 +360,14 @@ SYM_FUNC_START_LOCAL(__create_page_tables) adrp x5, __idmap_text_end clz x5, x5 cmp x5, TCR_T0SZ(VA_BITS_MIN) // default T0SZ small enough? - b.ge 1f // .. then skip VA range extension + b.ge 5f // .. then skip VA range extension adr_l x6, idmap_t0sz str x5, [x6] + cbnz x25, 4f // skip cache invalidation if MMU is on dmb sy dc ivac, x6 // Invalidate potentially stale cache line +4: #if (VA_BITS < 48) #define EXTRA_SHIFT (PGDIR_SHIFT + PAGE_SHIFT - 3) @@ -387,7 +394,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables) mov x4, #1 << (PHYS_MASK_SHIFT - PGDIR_SHIFT) str_l x4, idmap_ptrs_per_pgd, x5 #endif -1: +5: ldr_l x4, idmap_ptrs_per_pgd adr_l x6, __idmap_text_end // __pa(__idmap_text_end) @@ -407,6 +414,8 @@ SYM_FUNC_START_LOCAL(__create_page_tables) map_memory x0, x1, x5, x6, x7, x3, x4, x10, x11, x12, x13, x14 + cbnz x25, 4f // skip cache invalidation if MMU is on + /* * Since the page tables have been populated with non-cacheable * accesses (MMU disabled), invalidate those tables again to @@ -422,7 +431,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables) adrp x1, init_pg_end bl dcache_inval_poc - ret x28 +4: ret x28 SYM_FUNC_END(__create_page_tables) /* -- 2.30.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel